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Chapter I 

Introduction & Methodology 

The American Customer Satisfaction Index (ACSI) is the national indicator of customer evaluations of the 
quality of goods and services available to U.S. residents. It is the only uniform, cross-industry/government 
measure of customer satisfaction. Since 1994, the ACSI has measured satisfaction, its causes and 
effects, for seven economic sectors, 41 industries, more than 200 private sector companies, two types of 
local government services, the U.S. Postal Service, and the Internal Revenue Service. ACSI has 
measured more than 1 00 programs of federal government agencies since 1 999. This allows 
benchmarking between the public and private sectors and provides information unique to each agency on 
how activities that interface with the public affect the satisfaction of customers. The effects of satisfaction 
are estimated, in turn, on specific objectives, such as public trust. 

The ACSI is produced through a partnership of the University of Michigan Business School, CFI Group, 
and the American Society for Quality. 

Segment Choice 

A total of 1 0 groups, composed of eight program offices, ED Facts Coordinators, and Chief State School 
Officers, participated in the 2008 U.S. Department of Education Grantee Satisfaction Survey. All 10 
groups had also participated in the 2006 and 2007 studies. The chart below indicates the composition of 
survey respondents by program groups as a percentage of all respondents. 



Programs 



State Title V, Part A 
Directors 

Chief State School 6% 

Officers 
6% 



Directors of Adult 
Education and Literacy 
14% 



Career and Technical 
Education State 
Directors 
9% 

OPEPD - 
EDEN/EDFacfS 
Coordinators 
10 % 



State Directors 
Special Educat 
11 % 







State Educational 
Technology Directors 
13% 



Title III State Directors 
11 % 



Lead Agency Early 
Intervention Directors 
11 % 



State Title I Directors 
11 % 



N=322 
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C. Customer Sample and Data Collection 

The same programs that participated in 2006 and 2007 were included in the 2008 Grantee Survey. Each 
program provided a list of Directors from their program. Chief State School Officers were also included. 
ED provided a total of 570 e-mail contacts. Data were collected from April 1 5, 2008 through June 26, 
2008. Data collection was conducted primarily by e-mail. In order to increase response reminder e-mails 
were sent to non-responders and phone calls were also placed to non-responders where respondents 
were given the option to complete the survey via phone. A total of 362 responded to the invitation for a 
63.5% response rate. Thirty-five respondents indicated that they were not affiliated with one of the 
participating program offices within the last 12 months, and were therefore disqualified. Of those who 
responded and were qualified, 322 respondents provided valid responses. These are responses where at 
least two-thirds of the questions were answered. 

Response rates for each participating program for 2007 and 2008 are provided below. For most of the 
programs response rates dipped slightly from last year. However, as was the case last year all but two 
programs had response rates above 50%. 

Response Rate by Program 





2007 


2008 




Number of 
Responses 


Response Rate 


Number of 
Responses 


Response 

Rate 


State Title 1 Directors 


48 


89% 


34 


64% 


Title III State Directors 


45 


87% 


34 


65% 


State Educational Technology Directors 


41 


76% 


41 


57% 


Directors of Adult Education and Literacy 


48 


84% 


44 


77% 


Career and Technical Education State Directors 


42 


78% 


28 


52% 


State Title V, Part A Directors 


23 


43% 


20 


36% 


EDEN/EDFacfs Coordinators 


32 


62% 


33 


58% 


State Directors of Special Education 


38 


58% 


34 


59% 


Chief State School Officers 


19 


34% 


18 


33% 


Lead Agency Early Intervention Directors 


29 


50% 


36 


64% 



D. Questionnaire and Reporting 

The questionnaire used is shown in Appendix A. The core set of questions was developed in 2005 and 
has remained unchanged in each subsequent administration of the survey. Each program had the 
opportunity to include a set of questions specific to their program. Some programs chose to add or modify 
their custom questions in 2008. Changes to the questionnaire are noted and can be found with the 
questionnaire in Appendix A. 

Most of the questions in the survey asked the respondent to rate items on a 1 to 10 scale. However, 
open-ended questions were also included within the core set of questions, as well as open-ended 
questions designed to be program-specific. Appendix C contains tables that show scores for each 
question reported on a 0 to 100 scale. Results are shown in aggregate and by program. All verbatim 
responses are included in the back of the report in Appendix D, Verbatim Comments. Comments are 
separated by program. 
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Chapter II 
Survey Results 

A. Customer Satisfaction (ACSI) 

The Customer Satisfaction Index (CSI) is a weighted average of three questions: Q30, Q31 , and Q32, 
in the questionnaire in Appendix A. The questions are answered on a 1 to 10 scale and are converted to 
a 0 to 1 00 scale for reporting purposes. The three questions measure: Overall satisfaction (Q30); 
Satisfaction compared to expectations (Q31); and Satisfaction compared to an ‘ideal’ organization (Q32). 

The 2008 Customer Satisfaction Index (CSI) for the Department of Education grantees is 65. 

Satisfaction with ED is up two points from last year to reach its highest level since the measure with ED 
began in 2005. When considering the three questions separately, overall satisfaction with ED’s products 
and services reached a score of 70. ED meeting expectations scored 63 and compared to the ideal was 
rated 59. 



Customer Satisfaction Index 

2005 - 2008 



Customer Satisfaction 



How satisfied are you with 
ED’s products and services 



How well ED's products and 
services meet expectations 



How well ED compares with 
ideal products and services 




57 

57 



50 



60 



70 



80 



90 



100 



■ 2008 III2007 D2006 D2005 
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The chart below compares the satisfaction score of the U.S. Department of Education with satisfaction 
scores from other federal grant awarding agencies taken over the past three years and the most recent 
(December 2007) annual overall federal government average for benchmarking purposes. The U.S. 
Department of Education’s score is on the lower end of federal grantee satisfaction scores. ED is now 
only three points below the current federal government average. 



Satisfaction Benchmarks 



Corp. for National and Community Service - 
AmeriCorps VISTA Grantees 



Corp. for National and Community Service 
Grantees 



Corp. for National and Community Service - Learn 
and Serve America Grantees 



Corp. for National and Community Service - Senior 
Corps Grantees 

USDA Cooperative State Research, Education and 
Extension Service Grantees 



Latest Federal Government Average (Dec. 2007) 



Department of Education 



Corp. for National and Community Service - 
AmeriCorps State and National Grantees 
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Satisfaction was up two points at the aggregate level. With respect to program-level scores there has 
been some changes from last year as well. The chart below reflects the grantees’ 2008 Customer 
Satisfaction Index with the Department by program and compares current scores with those from 2007. 
As was the case in 2007, in 2008 three programs had statistically significant changes in their satisfaction 
with the Department among their Directors. The three programs are noted below with asterisks, 
EDEN/EDFacfs Coordinators, Chief State School Officers, and Title III State Directors. None of the other 
changes shown below, either gains or drops, were statistically significant at a 90% level of confidence. 

Customer Satisfaction Index 

Scores by Program 



Overall 



Directors of Adult Education and 
Literacy 

State Title V, Part A Directors 



* EDEN/EDFacts Coordinators 

Lead Agency Early Intervention 
Directors 

* Chief State School Officers 



State Title I Directors 

Career and Technical Education 
State Directors 

State Educational Technology 
Directors 

State Directors of Special 
Education 

* Title III State Directors 




* Statistically significant difference from 2007 scores at 90% level of confidence. 

For an explanation of significant differences in scores between years, see Appendix E. 
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B. Customer Satisfaction Model 

The government agency ACSI model is a variation of the model used to measure private sector 
companies. Both were developed at the National Quality Research Center of the University of Michigan 
Business School. Whereas the model for private sector, profit-making companies measures Customer 
Loyalty as the principal outcome of satisfaction (measured by questions on repurchase intention and price 
tolerance), each government agency defines the outcomes most important to it for the customer segment 
measured. Each agency also identifies the principal activities that interface with its customers. The model 
provides predictions of the impact of these activities on customer satisfaction. 

The U.S. Department of Education Grantee Customer Satisfaction model - illustrated below, should be 
viewed as a cause-and-effect model that moves from left to right, with satisfaction (ACSI) in the middle. 
The rectangles are multi-variable components that are measured by survey questions. The numbers in 
the upper right corners of the rectangles represent performance or attribute scores on a 0 to 100 scale. 
The numbers in the lower right corners represent the strength of the effect of the component on the left on 
the one to which the arrow points on the right. These values represent "impacts”. The larger the impact 
value, the more effect the component on the left has on the one on the right. The meanings of the 
numbers shown in the model are the topic of the rest of this chapter. 



2008 U.S. Department of Education Grantee Satisfaction Model 



Effectiveness in using technology to deliver services 72 
ED's automated process to share info 
Effectiveness in improving state’s reporting 
Expected reduction in federal paperwork 

Clarity 
Organization 
Sufficiency of detail 
Relevance 
Comprehensiveness 

Knowledge of legislation, regulations ... 

Responsiveness to questions 
Accuracy of responses 
Sufficiency of legal guidance 
Consistency of responses with ED staff 
Collaboration with other ED offices 

Knowledge of legislation, regulations ... 81 

Responsiveness to questions 84 

Accuracy of responses 32 

Sufficiencyof legal guidance 7^ 

Consistency of responses with ED staff 79 

Collaboration with ED staff in providing services 81 
Collaboration with other ED-Funded Tech Asst. 79 

Ease of finding materials 61 

Ease of submitting info to ED 74 



72 

69 

67 

59 


Technology 

1.2 


73 

76 




V 


71 

78 


Documents 


69 


1.4 






85 




80 

82 


ED Staff/ 


75 


Coordination 


72 

70 


0.9 



ED-Funded 
Tech. Asst 



3 

-0.3 



Online 
Resources 



3 

JOJ) 



Customer 

Satisfaction 

Index 



Overall Satisfaction 70 

Compared to expectations 63 
Compared to ideal 59 



2 % 
Complaints 






N=322 
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Attribute scores are the mean (average) respondent scores to each individual question in the survey. 
Respondents are asked to rate each item on a 1 to 10 scale, with “1" being “poor” and “10” being 
“excellent.” For reporting purposes, CFI Group converts the mean responses to these items to a 0 to 100 
scale. It is important to note that these scores are averages and not percentages. The score should be 
thought of as an index in which “0” represents “poor” and “100” represents “excellent.” 

A component score is the weighted average of the individual attribute ratings given by each respondent to 
the questions presented in the survey. A score is a relative measure of performance for a component, as 
given for a particular set of respondents. In the model illustrated on the previous page Clarity, 
Organization, Sufficiency of detail, Relevance, and Comprehensiveness are combined to create the 
component score for “Documents.” 

Impacts should be read as the effect on the subsequent component if the initial driver (component) were 
to be improved or decreased by five points. For example, if the score for “Documents” increased by 5 
points (73 to 78), the Customer Satisfaction Index would increase by the amount of its impact, 1 .4 points, 
(from 65 to 66.4). (Note: Scores shown are reported to nearest whole number). Similarly, if the Customer 
Satisfaction Index were to increase by 5 points, “Complaints” would decrease by 0.7%. If the driver 
increases by less than or more than five points, the resulting change in the subsequent component would 
be the corresponding fraction of the original impact. Impacts are additive. Thus, if multiple areas were 
each to improve by 5 points, the related improvement in satisfaction will be the sum of the impacts. 
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C. Drivers of Customer Satisfaction 

Technology 

Impact 1.2 

Technology continues to have a high impact on grantee satisfaction with an impact of 1 .2 on satisfaction. 
The area of technology is up two points from 2007 with a score of 67. This represents a statistically 
significant increase from last year. The U.S. Department of Education’s effectiveness in using technology 
to deliver its services remains the highest rated item in the area of technology at a rating of 72. The 
Department’s automated process to share accountability information, effectiveness in improving state’s 
reporting, and expected reduction in federal paperwork all had significant gains from last year. 

Technology 

Aggregate Scores 



* 

Technology 



ED’s effectiveness in using 
technology to deliver its 
services 



* ED's quality of assistance 



Effectiveness of automated 
process in improving 
state’s reporting 

* 

Expected reduction in 
federal paperwork 




* Statistically significant difference from 2007 scores at 90% level of confidence. 

For an explanation of significant differences in scores between years, see Appendix E. 

Respondents who rated “ED’s effectiveness in using technology to deliver services” low (below “6”) were 
asked how the U.S. Department of Education could better use technology to deliver its services. As was 
the case in previous years, many respondents mentioned increasing the use of conference calls and 
WebEx in order to promote better communication without the need for travel. Podcasting was also 
mentioned as a possible way to provide information to grantees. All verbatim responses can be found in 
Appendix D. 
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While at an aggregate level, grantees’ evaluation of Technology increased two points, when considering 
Technology scores for the U.S. Department of Education grouped by program, there are a few programs 
with more sizable and significant changes from last year. Three programs had a significant increase in 
their rating of Technology this year, Adult Education and Literacy, Chief State School Officers, and Title 
III. None of the programs rated Technology significantly lower in 2008. 

Technology 

Scores by Program 



* Overall 

Directors of Adult Education and Literacy 
EDEN/EDFacts Coordinators 
* Chief State School Officers 
State Title V, Part A Directors 
* Title III State Directors 
Career and Technical Education State Directors 
State Educational Technology Directors 
State Title I Directors 
Lead Agency Early Intervention Coordinators 
State Directors of Special Education 




□ 2008 0 2007 



* Statistically significant difference from 2007 scores at 90% level of confidence. 
Foran explanation of significant differences in scores between years, see Appendix E. 
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While ED’s effectiveness in using technology to deliver its services was the highest rated item overall in 
the area of Technology, the quality of assistance from the automated process to share accountability 
information was the highest rated item by four programs, State Educational Technology Directors, State 
Title V, Part A Directors, EDEN/EDFacfs Coordinators, and Directors of Adult Literacy. So for many 
programs this too is perceived as a strength for the Department. Expected reduction in federal paperwork 
was the item that had the greatest range of ratings. Chief State School Officers felt most positively about 
the paperwork reduction (75). However, Lead Agency Early Intervention Coordinators only rated this item 
39 and four other programs rated expected reduction in federal paperwork in the 50s. 



Technology 

Technology Attribute Scores by Program 





Chief State School Officers 


Title III State Directors 


State Educational 
Technology Directors 


State Title 1 Directors 


State Title V, Part A 
Directors 


Lead Agency Early 
Intervention Coordinators 


State Directors of Special 
Education 


EDEN/ED Facts 
Coordinators 


Career and Technical 
Education State Directors 


Directors of Adult Education 
and Literacy 


Technology 


72 


68 


65 


64 


69 


64 


59 


73 


66 


75 


ED’s effectiveness in using technology to deliver its services 


71 


70 


67 


67 


75 


77 


66 


75 


74 


76 


ED's quality of assistance 


73 


69 


70 


62 


75 


63 


57 


79 


66 


79 


Effectiveness of automated process in improving state’s reporting 


68 


69 


66 


63 


74 


63 


57 


72 


60 


75 


Expected reduction in federal paperwork 


75 


62 


64 


57 


51 


39 


52 


67 


56 


67 


Sample Size 


18 


34 


41 


34 


20 


36 


34 


33 


28 


44 
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Documents 

Impact 1.4 

Documents continues to be a key satisfaction driver with an impact of 1 .4. Performance in the area of 
Documents saw a sizeable and significant four-point increase. All of the items in this area had statistically 
significant increases over their 2007 ratings. Comprehensiveness in addressing the scope of issues that 
you face and sufficiency of detail to meet your program needs each improved by six points from last year. 

Documents 

Aggregate Scores 



1 Documents 



* 

Relevance to your areas of need 



* Organization of information 



* Clarity 



* 

Sufficiency of detail to meet your program needs 



* Comprehensiveness in addressing the scope of issues 
that you face 




N=322 



* Statistically significant difference from 2007 scores at 90% level of confidence. 

For an explanation of significant differences in scores between years, see Appendix E. 
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The aggregate increase in the rating of Documents is due to a broad increase in this area across most 
programs as a majority of the programs (six) had significant increases over last year’s ratings. No 
program has a statistically significant decrease, and only two of the programs rated Documents less than 



70. 



Documents 

Scores by Program 



* Overall 

Directors of Adult Education and Literacy 
State Title V, Part A Directors 

* 

Career and Technical Education State Directors 

State Title I Directors 
* EDEN/EDFacts Coordinators 

* 

State Educational Technology Directors 
State Directors of Special Education 

Lead Agency Early Intervention Coordinators 

* 

Chief State School Officers 
Title III State Directors 




* Statistically significant difference from 2007 scores at 90% level of confidence. 
For an explanation of significant differences in scores between years, see Appendix E. 
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Across most of the programs Documents received strong ratings for their relevance to the grantees needs 
and for their organization of information. Clarity and detail of the documents received solid ratings from 
most programs, although Chief State School Officers and Title III State Directors rated both areas in the 
60s. Programs were less uniform in their ratings of the Documents’ comprehensiveness in addressing the 
scope of issues that they face. Programs that rated Documents the highest, such as State Title V, Part A 
Directors and Directors of Adult Education and Literacy, gave strong ratings to this item. Conversely, 
those programs giving lower scores to Documents tended to rate this item lower. State Title III Directors 
rated comprehensiveness 59. State Directors of Special Education and Lead Agency Early Intervention 
Coordinators rated it 64 and 65, respectively. 



Documents 

Documents Attributes Scores by Program 





Chief State School Officers 


Title III State Directors 


State Educational 
Technology Directors 


State Title 1 Directors 


State Title V, Part A 
Directors 


Lead Agency Early 
Intervention Coordinators 


State Directors of Special 
Education 


EDEN/ED Facts 
Coordinators 


Career and Technical 
Education State Directors 


Directors of Adult Education 
and Literacy 


Documents 


69 


66 


73 


74 


77 


70 


71 


74 


76 


79 


Clarity 


66 


65 


72 


76 


77 


70 


70 


72 


75 


81 


Organization of information 


72 


69 


74 


75 


79 


72 


76 


76 


79 


82 


Sufficiency of detail to meet your program needs 


68 


63 


72 


71 


73 


67 


67 


72 


75 


79 


Relevance to your areas of need 


71 


73 


78 


79 


80 


78 


80 


80 


81 


80 


Comprehensiveness in addressing the scope of issues that you face 


67 


59 


70 


69 


77 


65 


64 


72 


69 


76 


Sample Size 


18 


34 


41 


34 


20 


36 


34 


33 


28 


44 
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ED Staff/Coordination 

Impact 0.9 

ED Staff/Coordination remains one of the higher-performing areas for the U.S. Department of Education 
and has improved by three points since last year. Its impact of 0.9 means that further improvements in 
this area will yield a modest increase in grantee satisfaction with the Department. All items in the area of 
Staff/Coordination had a statistically significant improvement over last year. Knowledge of relevant 
legislation, regulations, policies, and procedures improved by four points to a rating of 85. This is the 
highest rated item in the entire survey. Other ED Staff/Coordination items realized improvements of two to 
four points at the aggregate level. 



ED Staff/Coordination 

Aggregate Scores 



* ED Staff/Coordination 



* 

Knowledge of relevant legislation, regulations, policies, and 
procedures 

* Accuracy of responses 



* 

Responsiveness to your questions 



Sufficiency of legal guidance in responses 



Consistency of responses with ED staff from different program 

offices 



* Collaboration with other ED offices in providing relevant services 




N=321 



* Statistically significant difference from 2007 scores at 90% level of confidence. 

For an explanation of significant differences in scores between years, see Appendix E. 
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While overall ED Staff/Coordination had a three-point improvement from last year, three of the programs, 
Title I, Title III, and Chief State School Officers rated this area significantly higher in 2008. Only one 
program, Special Education rated ED Staff/Coordination significantly lower in 2008. None of the other 
programs saw significant changes from last year in the area of ED Staff/Coordination. 



ED Staff/Coordination 

Scores by Program 



Overall 

State Title V, Part A Directors 
Directors of Adult Education and Literacy 

State Educational Technology Directors 

* 

State Title I Directors 
Lead Agency Early Intervention Coordinators 
EDEN/EDFacts Coordinators 
Career and Technical Education State Directors 
Title III State Directors 
* Chief State School Officers 
* State Directors of Special Education 




* Statistically significant difference from 2007 scores at 90% level of confidence. 
Foran explanation of significant differences in scores between years, see Appendix E. 
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Across all programs, grantees are finding the Department’s Staff to be knowledgeable of relevant 
legislation, regulations, policies, and procedures. Ratings for this item range from a low of 80 to a high of 
90. Even State Directors of Special Education, who only rated ED Staff/Coordination 65 overall, rated 
knowledge 80. Accuracy of responses also yields high scores across nearly all programs. The areas of 
consistency of responses with ED Staff from different program offices and collaboration with other ED 
offices in providing relevant services were rated high by several programs including State Title V, Part A 
Directors, Lead Agency Early Intervention Coordinators, Directors of Adult Education and Literacy, and 
State Educational Technology Directors. Chief State School Officers, Title III State Directors, and State 
Directors of Special Education provided much lower ratings to consistency and collaboration. 



ED Staff/Coordination 

ED Staff/Coordination Attribute Scores by Program 





Chief State School Officers 


Title III State Directors 


State Educational 
Technology Directors 


State Title 1 Directors 


State Title V, Part A 
Directors 


Lead Agency Early 
Intervention Coordinators 


State Directors of Special 
Education 


EDEN/ED Facts 
Coordinators 


Career and Technical 
Education State Directors 


Directors of Adult Education 
and Literacy 


ED Staff/Coordination 


73 


75 


81 


81 


85 


80 


65 


80 


77 


83 


Knowledge of relevant legislation, regulations, policies, and procedures 


85 


82 


82 


90 


87 


85 


80 


84 


85 


87 


Responsiveness to your questions 


74 


83 


82 


79 


84 


82 


64 


85 


77 


86 


Accuracy of responses 


80 


79 


84 


90 


89 


81 


70 


84 


79 


84 


Sufficiency of legal guidance in responses 


69 


71 


82 


80 


88 


74 


58 


80 


72 


79 


Consistency of responses with ED staff from different program offices 


66 


64 


80 


74 


82 


78 


62 


69 


74 


76 


Collaboration with other ED offices in providing relevant services 


59 


63 


72 


67 


81 


79 


51 


73 


71 


81 


Sample Size 


18 


34 


41 


34 


20 


36 


34 


33 


28 


44 
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Online Resources 

Impact 0.9 

Online Resources had a three-point improvement in its score compared to last year. Ease of submitting 
information to ED via the Web received higher ratings in 2008 with a rating of 74. In this year’s customer 
satisfaction model it was found that the impact that Online Resources has on satisfaction is substantially 
greater than it was last year with an impact of 0.9. This means that future improvements in the area of 
Online Resources will yield a larger increase in customer satisfaction that what would have been 
previously expected. 



Online Resources 

Aggregate Scores 



Online Resources 




* Ease of submitting 
information to ED via the 
Web 



Ease of finding materials 
online 




N=318 
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Four programs rated Online Resources significantly higher in 2008 than they did in 2007. This includes 
Chief State School Officers, EDEN/EDFacfs, Adult Education and Literacy, and Title III. Conversely, 
Career and Technical Education rated the Department significantly lower on Online Resource in 2008. 



* Overall 



Lead Agency Early Intervention Coordinators 

* Chief State School Officers 

* 

EDEN/EDFacts Coordinators 
Directors of Adult Education and Literacy 

State Directors of Special Education 

* 

Title III State Directors 
State Title V, Part A Directors 
* Career and Technical Education State Directors 
State Educational Technology Directors 
State Title I Directors 



* Statistically significant difference from 2007 scores at 90% level of confidence. 

For an explanation of significant differences in scores between years, see Appendix E. 
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For most programs ease of submitting information to the Department via the Web received positive 
ratings. Only two programs, State Educational Technology Directors and State Title I Directors rated ease 
of submitting information below 70. Ease of finding materials online was a different matter. Only 
EDEN/EDFacfs Coordinators rated ease of finding materials as high as 70. State Title I Directors (47), 
Career and Technical Education State Directors (54), and State Title V, Part A Directors (57) found ease 
of finding materials online most problematic. 

Online Resources 

Online Resources Attribute Scores by Program 
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ED-funded Technical Assistance 

Impact 0.3 

ED-funded Technical Assistance remains the highest scoring area for the U.S. Department of Education. 
This year’s score reached 80 with a three-point improvement over last year’s score. Its relatively low 
impact of 0.3 means that a further improvement in ED-funded Technical Assistance will only yield a very 
modest increase in satisfaction. Five of the seven items in the area of ED-funded Technical Assistance 
had statistically significant gains from last year. ED-funded Technical Assistance was found to be 
responsive to questions, providing accurate responses, and knowledgeable of relevant legislation, 
regulations, policies, and procedures. Collaboration with ED staff in providing relevant services also 
received a strong rating (81). 

ED-funded Providers of Technical Assistance 

Aggregate Scores 



* 

ED-funded Technical Assistance 



* Responsiveness to your questions 

* 

Accuracy of responses 

* Knowledge of relevant legislation, regulations, policies, 

and procedures 

* Collaboration with ED staff in providing relevant services 

Consistency of responses with ED staff 

Collaboration with other ED-funded providers of 
technical assistance 

Sufficiency of legal guidance in responses 




* Statistically significant difference from 2007 scores at 90% level of confidence. 

For an explanation of significant differences in scores between years, see Appendix E. 
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Three programs rated ED-funded Providers of Technical Assistance significantly higher than they did in 
2007. This includes EDEN/EDFacfs, Lead Agency Early Intervention Coordinators, and Chief State 
School Officers. No other program had a statistically significant change in their rating of ED-funded 
Providers of Technical Assistance. 



ED-funded Providers of Technical Assistance 

Scores by Program 
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* Statistically significant difference from 2007 scores at 90% level of confidence. 

For an explanation of significant differences in scores between years, see Appendix E. 
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As was the case in 2007, ED-funded Providers of Technical Assistance received the strongest ratings for 
knowledge, responsiveness, and accuracy of responses across most programs with responsiveness 
receiving a rating of 92 from ED/EDFacfs Coordinators and Lead Agency Early Intervention Coordinators. 
Collaboration with ED staff in providing relevant services and collaboration with other ED-funded 
providers of technical assistance was solid across nearly all programs as well, with EDEN/EDFacfs 
Coordinators and Directors of Adult Education and Literacy rating Collaboration with ED staff 90. 
Sufficiency of legal guidance in the response continues to be the biggest issue for most programs with six 
programs rating sufficiency of legal guidance in the 60s. However, for State Educational Technology 
Directors, State Title V, Part A Directors, EDEN/EDFacfs Coordinators and Directors of Adult Education 
and Literacy guidance appears to be sufficient with ratings above 80 from these programs. 



ED-funded Providers of Technical Assistance 

ED-funded Providers of Technical Assistance Attribute Scores by Program 
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D. Satisfaction Benchmarks 

The satisfaction benchmark question “Overall, when I think of all of ED’s products and services, I am 
satisfied with their quality” was included in the 2008 survey. Respondents rated their satisfaction with all 
of the U.S. Department of Education’s products and services on a 4-point scale. In 2008, 83% responded 
‘Agree’ or ‘Strongly Agree.’ The percentage who answered ‘Agree’ or ‘Strongly Agree’ have remained 
relatively consistent over the course of the past four years only ranging from a low of 79% in 2006 to a 
high of 83% in both 2005 and 2008. 



Overall, when I think of all of ED’s products and services, I am satisfied with their quality. 



Strongly Agree 



Agree 



Disagree 



Strongly Disagree 



Does Not Apply 




□ 2008 

□ 2007 

□ 2006 
□ 2005 
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E. Complaints 

Only 2% of all respondents reported that they had formally complained to the U.S. Department of 
Education within the past six months. This is down from 3% reported last year. Only four programs even 
had any respondents who had formally complained in 2008. This is also a decrease in number from last 
year as in 2007 six programs had respondents who had complained. As was the case in 2007, Chief 
State School Officers were most likely to complain. However, complaints for this group were down from 
21% to 11%. 



Overall 



Chief State School Officers 

State Directors of Special 
Education 

* Title III State Directors 

Lead Agency Early Intervention 
Coordinators 

jj* 

State Educational Technology 
Directors 



State Title I Directors 
State Title V, Part A Directors 
EDEN/EDFacts Coordinators 



"Career and Technical Education 
State Directors 

Directors of Adult Education 
and Literacy 



Complaints 

Percentage by Program 





N=322 



* Statistically significant difference from 2007 scores at 90% level of confidence. 
Foran explanation of significant differences in scores between years, see Appendix E. 



2008 



26 



CFI 



Group 



Claes Fornell International 




Department of Education Office of the Chief Financial Officer Final Report 

Grantee Satisfaction Survey 

Chapter III 

Summary and Recommendations 

Grantee satisfaction with the Department improved by two points in 2008 to 65. This is only three points 
below the latest federal government average (68). Three programs, Directors of Adult Education and 
Literacy, State Title V, Part A Directors, and EDEN/EDFacfs Coordinators, have satisfaction with the 
Department above the federal government average. While a couple of programs, State Title I Directors 
and State Directors of Special Education rated satisfaction with the Department lower in 2008 than they 
did in 2007, these drops were not statistically significant. Three programs did have significant increases in 
satisfaction with the Department - Chief State School Officers, EDEN/EDFacfs, and Title III. Across all 
programs, complaints fell from 3% to 2%. 

All satisfaction driver areas saw significant increases from 2007. Documents had the largest increase 
(four points). Grantees continue to find the areas of ED Staff/Coordination and ED-funded Technical 
Assistance as the biggest strengths of the Department. Both ED Staff and ED-funded Technical 
Assistance were thought to be knowledgeable, provided grantees accurate responses, and were 
responsive to their questions. For ED Staff, the areas of consistency of responses with ED staff from 
different program offices, and collaboration with other ED offices in providing services, while still among 
the lower rated items were rated significantly higher in 2008. With ED-funded Technical Assistance the 
only area that is somewhat of an issue with grantees is the sufficiency of legal guidance received in 
responses. 

As was the case last year, Technology and Documents have the most impact on grantee satisfaction. 

This does not necessarily mean that these areas are the most import to grantees, but rather given the 
current level of performance across all areas, improvements in Technology and Documents will yield the 
largest increases in customer satisfaction. 

Technology and Online Resources remain the lower rated driver areas. With respect to the Department’s 
Online Resources, grantees were rather positive about the ease of submitting information to the 
Department via the Web. What was more problematic was finding materials online. Grantees continue to 
give solid ratings to the Department’s use of technology to deliver its services. Grantees’ ratings also 
showed improvements in the areas of working with states to develop an automated process to share 
accountability information and effectiveness of the automated process in improving state’s reporting. 

While the expected reduction in paperwork received higher ratings in 2008, it still remains the lowest 
rated item overall for the Department. 

Documents, the area with the highest impact on satisfaction, improved along all its attributes in 2008. 
Grantees felt the Documents were relevant to their areas of need, had information well organized and for 
the most part were clear. Comprehensiveness and detail, while improving six points in 2008, still remain 
as the lowest rated items for Documents. Further improvements in the detail that allows grantees to meet 
their programs’ needs and comprehensiveness in addressing issues they face will drive overall grantee 
satisfaction with the Department. 

Results by Program 

• Chief State School Officers had a significant increase in their satisfaction with the Department in 2008 
- up 12 points. Ratings across all of the Department performance areas were up for this group and 
complaints were down. Most performance areas including: ED Staff/Coordination, Online Resources, 
Technology, and Documents were rated in the 50s last year. This year all of those areas received ratings 
in the 70s or very near 70. (Documents was rated 69). One item of note, this group also continues to 
have the lowest response rate of any program with only 33% responding in 2008. 

• OELA Title III State Directors were the least satisfied with the Department in 2008 with satisfaction at 
59. However, they did rate satisfaction significantly higher in 2008 than they did in 2007. Significantly 
higher ratings in the areas of Documents, Technology, ED Staff/Coordination, and Online Resources 
drove this increase in satisfaction. Title III Directors gave positive ratings to OELA for providing them an 
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interpretation of Title III and helping them implement it in their state. However, the effectiveness of the 
relationship between Title I and Title III still receives low marks with respect to encouraging collaboration. 

• OESE State Education Technology Directors rated satisfaction higher in 2008, but not significantly so, 
and they continue to be among the least satisfied groups. Their evaluation of most of the driver areas did 
not change significantly from last year. However, they did rate Documents significantly higher with the 
biggest gains in the relevance and comprehensiveness of the Documents. State Education Technology 
Directors thought the relationship with the EETT program office was very effective. They found the 
guidance document provided by EETT office to be useful and that EETT was helpful in providing an 
interpretation and implementation of Title II, Part D. 

• OESE State Title I Directors rated their satisfaction with the Department lower in 2008 than they did in 
2007. However, it was not a statistically significant drop. State Title I Directors rated most of the driver 
areas the same in 2008 as they did in 2007. The only exception was ED Staff/Coordination, which was 
rated higher in 2008 with the biggest gains in ED Staff knowledge and accuracy of responses. In 2008, 
State Title I Directors found SASA to be more effective in using technology to enhance communication 
between the Department and the state. Information for monitoring for Title I was found to be available and 
useful. Technical Assistance on Even Start and Homeless Education also received strong ratings. 

• State Title V, Part A Directors gave the Department slightly lower satisfaction scores in 2008. 
However, the drop was not significant and they still remain among the most satisfied groups. There were 
no significant changes in their ratings of the driver areas. Title V, Part A Directors continued to give strong 
ratings to the Innovative Programs office for providing an interpretation and implementation of Title V, 

Part A. The guidance document and information at national meetings were found to be useful. The 
working relationship with the Title V, Part A program office was rated as being quite effective. 

• Lead Agency Early Intervention Coordinators continue to rate their satisfaction with the Department 
among the middle of the program groups. There was no change in their rating of satisfaction from last 
year. Their evaluation of driver areas did not show significant changes from last year with the exception of 
ED-funded Technical Assistance. Lead Agency Early Intervention Coordinators found ED-funded 
Technical Assistance to be more responsive, more accurate and gave higher ratings to their 
collaborations with ED staff and other ED-funded providers. OSEP’s Technical Assistance and 
Dissemination Centers were found to be even more responsive to answering questions in 2008 compared 
to 2007. Monitoring and State Improvement Planning Division Contacts also received high marks for 
responsiveness, supportiveness in helping complete performance plans, and dissemination of accurate 
information. 

• State Directors of Special Education rated their satisfaction with the Department among the lowest of 
the groups. In particular, State Directors of Special Education gave significantly lower ratings to the ED 
Staff/Coordination. This was driven by significant and sizable drops in the sufficiency of legal guidance, 
collaboration with other ED offices and responsiveness to their questions. Technical Assistance and 
Dissemination Centers received significantly higher marks for the timeliness of disseminating information 
in 2008. Centers also received strong ratings for their responsiveness to answering questions and 
information requests. 

• EDEN/EDFacfs Coordinators’ satisfaction with the Department increased significantly in 2008. 
EDEN/EDFacfs Coordinators are now among those groups most satisfied with the Department. Their 
increase in satisfaction was driven by higher ratings of Documents and Online Resources. ED-funded 
Technical Assistance, while not a key driver of satisfaction was also rated significantly higher by 
EDEN/EDFacfs Coordinators in 2008. Documents were found to have more relevance and be more 
comprehensive for the scope of issues they faced. EDEN/EDFacfscontinues to give very high ratings to 
the Partner Support Center for the support they provide and the Department’s EDEN/EDFacfs team for 
their support being timely, useful, relevant, and accurate. Some of the lower rated areas saw significant 
gains in 2008, including the data submission process helping them meet federal mandates for data 
collection and submission and helping to improve the state data collection and submission processes. 
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• OVAE Career and Technical Education State Directors satisfaction with the Department remains 
among the lowest of the programs. Online Resources was rated significantly lower in 2008 driven by a 
lack of ease of finding materials online. Conversely, Career and Technical Education State Directors 
rated Documents significantly higher in 2008 with the biggest gain in the sufficiency of detail in the 
Documents to meet their needs. Those who were monitored by OVAE within the past year, found the 
federal monitoring process effective in identifying and correcting compliance issues in their state and 
improving program quality. One area where there may be an opportunity to improve involves the 
compatibility with state reporting systems for both the Consolidated Annual Report and the state plan 
submission database. 

• Directors of Adult Education and Literacy rating of satisfaction with the Department was the highest of 
all programs. Three of the driver areas received significantly higher scores in 2008. Online Resources 
was up due to a higher rating of ease of submitting information via the Web. Technology was up mostly 
due to an improvement in the quality of assistance regarding the automated process in sharing 
accountability information. Documents were found to have more detail, be more comprehensive and have 
more clarity. Directors of Adult Education found the National Reporting System to be easy to use and the 
training to support it to be useful. National meetings and conferences provided by OVAE receive high 
marks for the relevance and usefulness of the information. 
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Recommendations 

While there have been sizeable improvements in 2008, the Department of Education should continue to 
focus on improving the high-impact, low-performing areas as first priorities. The grid below shows the 
performance and impact of each driver area. Technology and Online Resources are among the lower 
performing areas and both have considerable impacts on satisfaction. 



Performance and Impact of Driver Areas 




Performance scores for each of the areas are represented on the vertical axis. These are on a scale of 0 to 100 with 100 being 
the best possible score. The impact each area has on satisfaction is shown on the horizontal axis with the impact representing the 
expected improvement in satisfaction given a five-point improvement in that area. 

Within the area of Online Resources, improving the ease of finding materials online should be a high 
priority. While it appears to be an issue across most programs, State Title I Directors, Career and 
Technical Education State Directors, and State Title V, Part A Directors found it to be most problematic. 
Respondents thought the Department was doing better in using Technology in 2008. However, expected 
reduction in paperwork remains the lowest rated area and the relatively modest ratings for the automated 
process with respect to sharing accountability information and effectiveness in improving state’s reporting 
suggest that further improvements to the automated process may be opportunities to improve grantee 
satisfaction. 

Respondents’ scores indicate that they feel the Documents are clearer, more organized and more 
relevant. Comprehensive of the information in addressing the scope of issues grantees face may be a 
secondary area to target for improvement. 

The areas of ED Staff/Coordination and ED-funded Technical Assistance remain strengths, which saw 
further improvements in 2008 and the current levels of performance should be maintained in these areas. 
Continue to build on the gains in consistency of responses and collaboration with the other Department 
offices. 
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APPENDIX A : SURVEY QUESTIONNAIRE 
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U.S. Department of Education 
Grantee Satisfaction Survey 2008 



I ntroduction 

The Department of Education (ED) is committed to serving and satisfying its customers. To this end, we 
have commissioned the CFI Group , an independent third-party research group, to conduct a survey that 
asks about your satisfaction with ED’s products and services and about ways that we can improve our 
service to you. 

The CFI Group and the Department of Education will treat all information in a secure fashion and will only 
provide aggregate results to Department personnel. All information you provide will be combined with 
information from other respondents for research and reporting purposes. Your individual responses will 
not be released. This brief survey will take about 15 minutes of your time. 

If you have any questions about this survey, please contact Jeanne Nathanson, 202-401 -061 8. 
Jeanne.Nathanson@ed.gov. 

This interview is authorized by the U.S. Office of Management and Budget Control No. 1 800-001 1 . 

Please note that ALL questions on this survey (unless noted otherwise) refer to your experiences over the 
PAST 12 MONTHS. 

Please click on the "Next" button below to begin the survey. 



Program Office 

Q1 . Please indicate your current program office. 

1 . Chief State School Officers (ASK CSSOI .) 

2. OELA- Title III State Directors (SKIP TO STAFF1) 

3. OESE - State Educational Technology Directors (SKIP TO STAFF1) 

4. OESE - State Title I Directors (SKIP TO STAFF1) 

5. OESE - State Title V, Part A Directors (SKIP TO STAFF1) 

6. OSERS/OSEP - Lead Agency Early Intervention Coordinators (SKIP TO STAFF1) 

7. OSERS/OSEP - State Directors of Special Education (SKIP TO STAFF1 ) 

8. OPEPD - EDEN/EDFacfs Coordinators (SKIP TO STAFF1) 

9. OVAE - Career and Technical Education State Directors (SKIP TO STAFF1) 

1 0. OVAE - Directors of Adult Education and Literacy (SKIP TO STAFF1 ) 

1 1 . None of the above currently applies (SKIP TO END) 
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ED Staff /Coordination 

(ASK Q2 Only if Q1= 1. Chief State School Officers) 

Q2. Do you have regular contact with a senior ED officer who can respond to your policy and 
programmatic questions? 

1 . Yes 

2. No 

Please think about the interactions you have had with senior ED officers and/or other ED staff. 

PLEASE NOTE: This does not include ED-funded technical assistance providers, such as regional labs, 
national associations, contractors, etc. 

(ALL PROGRAMS OTHER THAN CHIEF STATE SCHOOL OFFICES START WITH Q3) 

On a scale from 1 to 10, where “1” is “Poor” and “10” is “Excellent,” please rate the senior ED officers’ 
and/or other ED staff’s: 

Q3. Knowledge of relevant legislation, regulations, policies, and procedures 

Q4. Responsiveness to your questions 

Q5. Accuracy of responses 

Q6. Sufficiency of legal guidance in responses 

Q7. Consistency of responses with ED staff from different program offices 
Q8. Collaboration with other ED program offices in providing relevant services 

(Ask Q9 only if Q8 is rated <6) 

Q9. Please identify your state’s best example of collaboration across offices that you would offer as a 
model for ED. 



ED-funded Technical Assistance 

Q10. Do you have interaction with ED-funded providers of technical assistance (e.g., regional labs, 
national associations, contractors, etc.) separate from ED staff? 

1 . Yes 

2. No (SKIP TO WEB 1.) 

3. Don’t know (SKIP TO WEB 1.) 

Please think about your interactions with ED-funded providers of technical assistance. On a 10-point 
scale, where “1” is “Poor” and “10” is “Excellent,” please rate their: 

Q1 1 . Knowledge of relevant legislation, regulations, policies, and procedures 

Q1 2. Responsiveness to your questions 

Q13. Accuracy of responses 

Q14. Sufficiency of legal guidance in responses 

Q15. Consistency of responses with ED staff 

Q16. Collaboration with ED staff in providing relevant services 

Q17. Collaboration with other ED-funded providers of technical assistance in providing relevant services. 
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Online Resources 

Please think about your experience using ED’s online resources. On a 1 0-point scale, where “1 ” is “Poor” 
and “10” is “Excellent,” please rate the: 

Q1 8. Ease of finding materials online 

Q1 9. Ease of submitting information to ED via the Web (e.g., grant applications, annual reports, 
accountability data) 



Technology 

Q20. Now think about how ED uses technology (e.g., conference calls, video-conferencing, Web 

conferencing, listservs) to deliver its services to you. On a 10-point scale, where “1” is “Not very 
effective” and “10” is “Very effective,” please rate ED’s effectiveness in using technology to deliver 
its services. 

(Ask Q21 only if Q20 is rated<6) 

Q21 . Please describe how ED could better use technology to deliver its services. 

Q22. Think about how ED is working with the states to develop an automated process to share 

accountability information. Please rate the quality of this assistance from ED. Use a 10-point scale 
where “1” is “Poor” and “10” is “Excellent.” 

Q23. How effective has this automated process been in improving your state’s reporting? Please use a 
10-point scale where “1” is “Not very effective” and “10” is “Very effective.” 

Q24. How much of a reduction in federal paperwork do you expect over the next few years because of 
ED’s initiative to promote the use of technology in reporting accountability data (e.g. 
EDEN/EDFacfs)? Please use a 10-point scale where “1” is “Not very significant” and “10” is “Very 
significant.” 



Documents 

Think about the documents (e.g., publications, guidance, memoranda) you receive from ED. 
On a 10-point scale, where “1” is “Poor” and “10” is “Excellent, please rate the documents’: 

Q25. Clarity 

Q26. Organization of information 

Q27. Sufficiency of detail to meet your program needs 

Q28. Relevance to your areas of need 

Q29. Comprehensiveness in addressing the scope of issues that you face 
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ACSI Benchmark Questions 

Now we are going to ask you to please consider ALL of ED’s products and services and not only those 

we just asked about. 

Q30. Using a 10-point scale on which “1” means “Very Dissatisfied” and “10” means “Very Satisfied,” how 
satisfied are you with ED’s products and services? 

Q31 . Now please rate the extent to which the products and services offered by ED have fallen short of or 
exceeded your expectations. Please use a 10-point scale on which "1" now means "Falls Short of 
Your Expectations" and "10" means "Exceeds Your Expectations." 

Q32. Now forget for a moment about the products and services offered by ED, and imagine the ideal 
products and services. How well do you think ED compares with that ideal? Please use a 10-point 
scale on which "1" means "Not Very Close to the Ideal" and "10" means "Very Close to the Ideal." 



Now please indicate the degree to which you agree or disagree with the following statement. 
Q33. Overall, when I think of all of ED’s products and services, I am satisfied with their quality. 



1 . 


Strongly Agree 


2. 


Agree 


3. 


Disagree 


4. 


Strongly Disagree 


5. 


Does Not Apply 


Closing 


Q34. In the past 6 months, have you issued a formal complaint to ED to express your dissatisfaction with 
the assistance you’ve received from an ED staff member? 


1 . 


Yes 


2. 


No 



Q35. Finally, please describe how ED can improve its service to you. 

Thank you again for your time. To complete the survey and submit the results, please hit the “Finish” 
button below. Have a good day! 
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2008 - OELA - Title III State Directors Questions 



Custom Questions - OELA - Title III State Directors 



Think about the particular ways in which you have received technical assistance from the Office of 
English Language Acquisition (OELA). 



Think about the one-on-one consultations you have had with program officers. On a 1 0-point scale, where 
“1” is “Not very effective” and “10” is “Very effective,” please rate the effectiveness of the one-on-one 
consultations in: 

Q1 . Providing you an interpretation of Title III 

Q2. Helping you with your implementation of Title III in your state 

Q3. What can OELA do over the next year to meet your state’s technical assistance and program 
improvement needs? 



Think about your experiences seeking information at OELA’s Clearinghouse Web site 
(www.ncela.gwu.edu). On a 1 0-point scale, where “1 ” is “Not very effective” and “1 0” is “Very effective,” 
please rate the effectiveness of the Web site in: 

Q4. Providing you with the information you needed 

Q5. Helping you inform programs serving ELLs in your state 

Q6. Think about the working relationship between Title III and Title I . 

On a 1 0-point scale, where “1 ” is “Not very effective” and “1 0” is “Very effective,” please rate how 
effective the Department has been in encouraging collaboration between Title I and Title III. 

(Ask only if question is scored <6) 

Q7. Please describe how the working relationship between Title III and Title I could be improved. 



Think about the Title III Biennial Report that is being used to collect data this cycle and the role of the 
regional meetings. On a 10-point scale, where “1” is “Poor” and “10” is “Excellent,” please rate the 
regional meetings for: 

Q8. Helping familiarize you with the Biennial Report form 

Q9. Allowing for your input and comments for refining the Biennial Report form 



Changes from 2007: None 
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2008 - OESE - STATE EDUCATIONAL TECHNOLOGY 

DIRECTORS 



Custom Questions - OESE - State Educational Technology Directors 



Think about the particular ways in which you have received technical assistance from the Enhancing 
Education Through Technology Program (EETT). 

First, consider the one-on-one consultations with EETT program officers. On a 10-point scale, where “1” 
is “Not very effective” and “1 0” is “Very effective,” please rate the effectiveness of the one-on-one 
consultations in: 

Q1 . Providing you an interpretation of Title II, Part D (Enhancing Education Through Technology) 

Q2. Helping you with your implementation of Title II, Part D (Enhancing Education Through 

Technology) 

Think about the guidance document provided by the EETT program office. 

Q3. On a 1 0-point scale, where “1 ” is “Poor” and “1 0” is “Excellent,” please rate its usefulness. 

Think about the Educational Technology State Directors' national meetings (i.e., national technology 
conferences, SETDA meetings) where the EETT program office made a presentation 

Q4. On a 1 0-point scale, where “1 ” is “Poor” and “1 0” is “Excellent,” please rate the usefulness of the 
information presented at these meetings. 



Think about the federal monitoring process as it relates to the Enhancing Education Through Technology 
program office. On a 10-point scale, where “1” is “Not very effective” and “10” is “Very effective,” please 
rate the effectiveness of the federal monitoring process in: 

Q5. Helping you with your compliance efforts 

Q6. Helping you to improve performance results 



Think about your working relationship with the Enhancing Education Through Technology program office. 

Q7. On a 1 0-point scale, where “1 ” is “Not very effective” and “1 0” is “Very effective,” please rate the 
effectiveness of this relationship. 

(Ask only if question is scored <6) 

Q8. Please describe how your working relationship with EETT could be improved. 

Q9. What can EETT do over the next year to meet your state’s technical assistance and program 

improvement needs? 



Changes from 2007: None 
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2008 - OESE - STATE TITLE I DIRECTORS 



Custom Questions - OESE - State Title I Directors 



Think about the technical assistance you have received from the Title I office, Student Achievement and 
School Accountability (SASA). On a 10-point scale, where “1” is “Poor” and “10” is “Excellent,” please rate 
the: 

Q1 . Usefulness of technical assistance on Standards and Assessments, Instructional Support, and 
Fiduciary of Title I, Part A of NCLB 

Q2. Usefulness of technical assistance on Neglected and Delinquent 

Q3. Usefulness of technical assistance on Even Start 

Q4. Usefulness of technical assistance on Homeless Education 



Think about the information on monitoring for Title I you have received. On a 1 0-point scale, where “1 ” is 

“Poor” and “10” is “Excellent,” please rate the: 

Q5. Availability of information on monitoring for Title I 

Q6. Usefulness of information on monitoring for Title I 

Q7. Think about how SASA uses electronic communications approaches such as email, Web casts, 
and WebEx to provide you information. On a 10-point scale, where “1” is “Not very effective” and 
“10” is “Very effective,” please rate SASA’s effectiveness in using technology to provide 
information. 

(Ask only if question is scored <6) 

Q8. Please describe how SASA could better use technology to provide information. 

Q9. Again, thinking about SASA’s use of electronic communications approaches to provide 

information: on a 1 0-point scale, where “1” is “Not very effective” and “10” is “Very effective,” 
please rate SASA’s effectiveness in using technology to enhance communication between ED 
and the state. 

Q1 0. What can SASA do over the next year to meet your state’s technical assistance and program 
improvement needs? 



Changes from 2007: Q7 and Q9 wording revised 
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2008 - OESE - STATE TITLE V, PART A DIRECTORS 

Custom Questions - OESE - State Title V, Part A Directors 
(Innovative Programs) 



Think about the particular ways in which you have received technical assistance from the Title V, Part A 
(Innovative Programs) office. First, consider the one-on-one consultations with Title V, Part A program 
officers. 

On a 1 0-point scale, where “1 ” is “Not very effective” and “1 0” is “Very effective,” please rate the 
effectiveness of the one-on-one consultations in: 

Q1 . Providing you an interpretation of Title V, Part A 
Q2. Helping you with your implementation of Title V, Part A 



Q3. Think about the guidance document provided by the Title V, Part A program office. On a 1 0-point 
scale, where “1” is “Poor” and “10” is “Excellent,” please rate its usefulness. 

Q4. Think about Title V, Part A national meetings and conference calls (including the Steering 
Committee’s national meetings and the program office’s conference calls for orientation and 
follow-up to the Steering Committee’s national meetings) where the Title V, Part A program office 
made presentations. On a 10-point scale, where “1” is “Poor” and “10” is “Excellent,” please rate 
the usefulness of the information presented by the program office. 



Think about the federal monitoring process as it relates to the Title V, Part A program office. On a 1 0- 
point scale, where “1” is “Not very effective” and “10” is “Very effective,” please rate the effectiveness of 
the federal monitoring process in: 

Q5. Helping you with your compliance efforts 

Q6. Helping you to improve performance results 



Q7. Think about your working relationship with the Title V, Part A program office. On a 1 0-point scale, 
where “1” is “Not very effective” and “10” is “Very effective,” please rate the effectiveness of this 
relationship. 

(Ask only if question is scored <6) 

Q8. Please describe how your working relationship with the Title V, Part A program office could be 
improved. 

Q9. What can the Title V, Part A program office do over the next year to meet your state’s technical 
assistance and program improvement needs? 

Changes from 2007: None 
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2008 - OSERS/OSEP - LEAD AGENCY EARLY INTERVENTION 

COORDINATORS 



Custom Questions - OSERS/OSEP - Lead Agency Early Intervention Coordinators 



Think about the technical support State Contacts from the Monitoring and State Improvement Planning 
Division of the Office of Special Education Programs provided. On a 10-point scale, where “1” is “Poor” 
and “10” is “Excellent,” please rate the staff’s: 

Q1 . Responsiveness to answering questions 

Q2. Supportiveness in helping you complete your state’s federally required performance 
plans/reports/applications 

Q3. Dissemination of accurate information 

Q4. Dissemination of information in a timely manner 



Think about the Technical Assistance and Dissemination Centers from OSEP. On a 1 0-point scale, where 
“1” is “Poor” and “10” is “Excellent," please rate the centers’: 

Q5. Responsiveness to answering questions 

Q6. Responsiveness to information requests 

Q7. Support to positively impact on your State’s SPP improvement targets. 



Q8. Think about the Communities of Practice from OSERS. On a 1 0-point scale, where “1 ” is “Not 
very effective” and “10” is “Very effective,” please rate its effectiveness in addressing systems 
improvement issues of the state. 

Q9. What can OSEP do over the next year to meet your state’s technical assistance and program 
improvement needs? 



Changes from 2007: added Q7 
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2008 - OSERS/OSEP - State Directors of Special Education 



Custom Questions - OSERS/OSEP -State Directors of Special Education 



Think about the technical support State Contacts from the Monitoring and State Improvement Planning 
Division of the Office of Special Education Programs provided. On a 10-point scale, where “1” is “Poor” 
and “10” is “Excellent,” please rate the staff’s: 

Q1 . Responsiveness to answering questions 

Q2. Supportiveness in helping you complete your state’s federally required performance 
plans/reports/applications 

Q3. Dissemination of accurate information 

Q4. Dissemination of information in a timely manner 



Think about the Technical Assistance and Dissemination Centers from OSEP. On a 1 0-point scale, where 

“1” is “Poor” and “10” is “Excellent,” please rate the centers’: 

Q5. Responsiveness to answering questions 

Q6. Responsiveness to information requests 

Q7 Support to positively impact on your State’s SPP improvement targets 

Q8. Think about the Communities of Practice from OSEP. On a 1 0-point scale, where “1 ” is “Not very 
effective” and “10” is “Very effective,” please rate its effectiveness in addressing systems 
improvement issues of the state. 

Q9. What can OSEP do over the next year to meet your state’s technical assistance and program 
improvement needs? 



Changes from 2007: Intro to Q1 -4 changed, Q7 added, Q8 OSERS changed to OSEP 
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2008 - OPEPD - EDEN/EDFACTS COORDINATORS 



Custom Questions - EDEN/EDFacts Coordinators 

Think about the support provided by the U.S. Department of Education EDEN/EDFacfs team. On a 1 0- 
point scale, where “1” is “Poor” and “10” is “Excellent,” please rate the: 

Q1. Timeliness of the support 

Q2. Usefulness of the support 

Q3. Accuracy of information 

Q4. Relevance of information 



Think about the EDEN/EDFacfs data submission process. On a 1 0-point scale, where “1 ” is “Not very 
effective” and “10” is “Very effective," please rate the effectiveness of the data submission process in: 

Q5. Helping you to meet federal mandates for data collection and submission 

Q6. Helping you to streamline your federal data collection and submission processes 

Q7. Helping you to improve state data collection and submission processes 



Q8. How much of a reduction in federal paperwork do you expect over the next few years because of 
the EDEN data submission process? Please use a 10-point scale where “1” is “Not very 
significant” and “10” is “Very significant.” 

Q9. How much do you expect the data you provide to contribute to improving education performance 
measurement? Please use a 1 0-point scale where “1 ” is “Not very significant” and “1 0” is “Very 
significant.” 

Q1 0. Think about the training provided by the EDEN/EDFacfs team on data submission. On a 1 0-point 
scale, where “1” is “Poor” and “10” is “Excellent,” please rate the training’s usefulness. 

Q1 1 . On a 1 0-point scale where “1 ” is”, Poor” and “1 0” is “Excellent,” please rate the support provided 
by the Department’s Partner Support Center. 

Q1 2. What has been the most significant change to your state data collection and submission process 
as a result of the ED Facts work? 

Q1 3. How can the Department’s ED Facts team be most helpful to you in meeting federal mandates for 
data collection, submission, analysis, and reporting in the coming year? 



Changes From 2007: None 
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2008- OVAE - CAREER AND TECHNICAL STATE DIRECTORS 



Custom Questions - OVAE - Career and Technical State Directors 

Think about the Consolidated Annual Report (CAR) as a way to report your state’s performance data to 
OVAE. On a 10-point scale, where “1” is “Poor” and “1 0” is “Excellent,” please rate the CAR’s: 

Q1. User-friendliness 

Q2. Compatibility with state reporting systems 



If you were monitored by OVAE within the last year, think about the federal monitoring process as it 
relates to your Perkins grant. On a 10-point scale, where “1” is “Not very effective” and “10” is “Very 
effective,” please rate the effectiveness of the federal monitoring process in: 

Q3. Identifying and correcting compliance issues in your state 
Q4. Helping you to improve program quality 

Q5. Think about the national leadership conferences and institutes offered by OVAE last year (i.e., 
Data Quality Institute in Savannah, GA, and Programs of Study Institutes in Chicago, IL, and 
Washington, DC). On a 10-point scale, where “1” is “Poor” and “10” is “Excellent,” please rate the 
effectiveness of these sessions on helping you to improve the quality of your programs and 
accountability systems. 

Q6. Think about the monthly Up-to-Date with DATE e-mails that are sent to you from OVAE. On a 
10-point scale, where “1” is “Not very effective” and “10” is “Very effective,” please rate the 
effectiveness of these e-mails in keeping you informed about key issues pertaining to all aspects 
of your Perkins grant (i.e., CAR reporting, State Plan submissions). 

Q7. Think about the Peer Collaborative Resource Network (PCRN) as it concerns OVAE. On a 1 0- 
point scale, where “1” is “Poor” and “10” is “Excellent,” please rate PCRN’s usefulness to your 
program. 

If you used the state plan submission database last year, think about this process as a way of submitting 
your five-year state plan to OVAE. (If you did not use the state plan submission database please select 
“N/A.”) On a 10 point scale, where “1” is Poor” and “10” is Excellent,” please rate the database on its: 



Q8. User-friendliness 

Q9. Compatibility with state reporting systems 



Q1 0. What can OVAE do over the next year to meet your state’s technical assistance and program 
improvement needs? 

Changes From 2008: Q5 and Q6 reworded, Q7 from 2007 dropped 
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2008 OVAE - DIRECTORS OF ADULT ED AND LITERACY 



Custom Questions - OVAE - Directors of Adult Ed and Literacy 

Think about the National Reporting System as a way to report your state’s performance data to OVAE. 

On a 1 0-point scale, where “1” is “Poor” and “10” is “Excellent,” please rate the NRS’s: 

Q1 . Ease of reporting using the NRS Web-based system. 

Q2. Think about the training offered by OVAE through its contract to support the National Reporting 
System (NRS). On a 10-point scale, where “1” is “Poor” and “10” is “Excellent,” please rate the 
usefulness of the training. 

If you have been monitored, think about the federal monitoring process as it relates to your AEFLA grant. 
On a 1 0-point scale, where “1 ” is,” Not Very Effective” and “1 0” is “Very effective,” please rate the 
effectiveness of the federal monitoring process on the following: 

Q3. Being well-organized 

Q4. Providing pre-planning adequate guidance 

Q5. Setting expectations for the visit. 

Q6. Using state peer reviewers in the federal monitoring process. 

Think about the national meetings and conference offered by OVAE. On a 1 0-point scale, where “1 ” is 
“Poor” and “10” is “Excellent”, please rate the information provided at these conference and institutes on 
the following: 

Q7. Being up-to-date 

Q8. Relevance of information 

Q9. Usefulness to your program 

Think about the national activities offered by DAEL. On a 10-point scale, where “1” is,” Poor” and “10” is 
“Excellent,” please rate the activities on the following: 

Q1 0. Usefulness of the products in helping your state meet AEFLA program priorities. 

Q1 1 . How well the technical assistance provided through the national activities address your program 
priorities and needs? Please use a 10-point scale where “1” means “does not address needs very 
well” and “10” means “addresses needs very well.” 

Q1 2. What can DAEL do over the next year to meet your state’s technical assistance/program 
improvement needs? 

Changes From 2007: None 
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2005 


2006 


2007 


2008 


Please indicate your current program office 


Percent 


Frequency 


Percent 


Frequency 


Percent 


Frequency 


Percent 


Frequency 


Chief State School Officers 


7% 


22 


8% 


28 


5% 


19 


6% 


18 


Title III State Directors 


12% 


40 


12% 


43 


12% 


44 


11% 


34 


State Educational Technology Directors 


14% 


47 


11% 


39 


11% 


40 


13% 


41 


State Title 1 Directors 


13% 


42 


13% 


46 


13% 


48 


11% 


34 


State Title V, Part A Directors 


- 


- 


9% 


33 


6% 


21 


6% 


20 


Lead Agency Early Intervention Directors 


8% 


26 


5% 


18 


8% 


29 


11% 


36 


State Directors of Special Education 


13% 


44 


10% 


36 


11% 


38 


11% 


34 


OPEPD - EDEN/EDFacfs Coordinators 


9% 


30 


9% 


33 


8% 


30 


10% 


33 


Career and Technical Education State Directors 


11% 


38 


10% 


37 


12% 


41 


9% 


28 


Directors of Adult Education and Literacy 


13% 


44 


15% 


56 


13% 


47 


14% 


44 


Have contact with a senior ED officer 










Yes 


86% 


19 


89% 


25 


79% 


15 


89% 


16 


No 


14% 


3 


11% 


3 


21% 


4 


11% 


2 


Have interaction with ED-funded providers of 


















technical assistance separate from ED staff 


















Yes 


85% 


282 


76% 


279 


78% 


280 


80% 


258 


No 


14% 


46 


23% 


84 


20% 


70 


18% 


59 


Don't Know 


2% 


5 


2% 


6 


2% 


7 


2% 


5 


Overall, when 1 think of all of ED’s products and 


















services, 1 am satisfied with their quality 


















Strongly Agree 


14% 


47 


11% 


40 


13% 


47 


15% 


49 


Agree 


69% 


228 


68% 


252 


68% 


243 


68% 


220 


Disagree 


15% 


49 


18% 


66 


14% 


51 


12% 


39 


Strongly Disagree 


2% 


7 


2% 


6 


2% 


6 


3% 


8 


Does Not Apply 


1% 


2 


1% 


5 


3% 


10 


2% 


6 


Issued a formal complaint about assistance received 


















from ED staff member 


















Yes 


3% 


9 


3% 


12 


3% 


9 


2% 


6 


No 


97% 


324 


97% 


357 


98% 


348 


98% 


316 




333 


369 


357 


322 
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Aggregate Scores 

Results compared to 2007 scores 





2007 Scores 


2008 Scores 


Significant Difference 


Customer Satisfaction Index 


63 


65 


s 


How satisfied are you with ED's products and services 


68 


70 


V 


How well ED's products and services meet expectations 


61 


63 


V 


How well ED compares with ideal products and services 


58 


59 





ED Staff/Coordination 


75 


78 


■/ 




Knowledge of relevant legislation, regulations, policies, and procedures 


81 


85 


V 




Responsiveness to your questions 


77 


80 


V 




Accuracy of responses 


79 


82 


V 




Sufficiency of legal guidance in responses 


73 


75 


V 




Consistency of responses with ED staff from different program offices 


68 


72 


V 




Collaboration with other ED offices in providing relevant services 


66 


70 


V 




ED-funded Technical Assistance 


77 


80 


•/ 




Knowledge of relevant legislation, regulations, policies, and procedures 


78 


81 


V 




Responsiveness to your questions 


81 


84 


V 




Accuracy of responses 


80 


82 


V 




Sufficiency of legal guidance in responses 


72 


71 






Consistency of responses with ED staff 


75 


79 


V 




Collaboration with ED staff in providing relevant services 


78 


81 


V 




Collaboration with other ED-funded providers of technical assistance 


77 


79 






Online Resources 


64 


67 


V 




Ease of finding materials online 


60 


61 






Ease of submitting information to ED via the web 


70 


74 


V 




Technology 


65 


67 


V 




ED’s effectiveness in using technology to deliver its services 


71 


72 






ED's quality of assistance 


66 


69 


V 




Effectiveness of automated process in improving state’s reporting 


64 


67 


V 




Expected reduction in federal paperwork 


56 


59 


V 





Documents 


69 


73 


■/ 


Clarity 


69 


73 


V 


Organization of information 


72 


76 


V 


Sufficiency of detail to meet your program needs 


65 


71 


V 


Relevance to your areas of need 


73 


78 


V 


Comprehensiveness in addressing the scope of issues that you face 


63 


69 


V 



Complaint 


3% 


2% 


Issued a formal complaint about assistance received from ED staff member 


3% 


2% 



Sample Size 



357 322 
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Aggregate Scores 

Results from 2005 through 2008 



2005 Score 


2006 Score 


2007 Score 


2008 Score 



Customer Satisfaction Index 63 62 63 65 



How satisfied are you with ED's products and services 


69 


67 


68 


70 


How well ED’s products and services meet expectations 


61 


60 


61 


63 


How well ED compares with ideal products and services 


57 


57 


58 


59 


ED Staff/Coordination 


76 


75 


75 


78 


Knowledge of relevant legislation, regulations, policies, and procedures 


83 


81 


81 


85 


Responsiveness to your questions 


76 


75 


77 


80 


Accuracy of responses 


81 


79 


79 


82 


Sufficiency of legal guidance in responses 


75 


72 


73 


75 


Consistency of responses with ED staff from different program offices 


70 


70 


68 


72 


Collaboration with other ED offices in providing relevant services 


63 


65 


67 


70 


ED-funded Technical Assistance 


79 


75 


77 


80 


Knowledge of relevant legislation, regulations, policies, and procedures 


79 


76 


78 


81 


Responsiveness to your questions 


82 


81 


81 


84 


Accuracy of responses 


80 


78 


80 


82 


Sufficiency of legal guidance in responses 


73 


69 


72 


71 


Consistency of responses with ED staff 


78 


74 


76 


79 


Collaboration with ED staff in providing relevant services 


80 


75 


78 


81 


Collaboration with other ED-funded providers of technical assistance 


78 


73 


77 


79 



Online Resources 


68 


65 


64 


67 


Ease of finding materials online 


64 


59 


60 


61 


Ease of submitting information to ED via the web 


72 


71 


70 


74 



Technology 


65 


64 


65 


67 


ED's effectiveness in using technology to deliver its services 


69 


71 


71 


72 


ED’s quality of assistance 


67 


66 


66 


69 


Effectiveness of automated process in improving state’s reporting 


61 


62 


64 


67 


Expected reduction in federal paperwork 


58 


54 


56 


59 



Documents 


71 


70 


69 


73 


Clarity 


71 


70 


69 


73 


Organization of information 


73 


73 


72 


76 


Sufficiency of detail to meet your program needs 


69 


67 


65 


71 


Relevance to your areas of need 


75 


73 


73 


78 


Comprehensiveness in addressing the scope of issues that you face 


67 


65 


63 


69 



Complaint 


3% 


3% 


3% 


2% 


Issued a formal complaint about assistance received from ED staff member 


3% 


3% 


3% 


2% 



Sample Size 



333 369 357 



322 
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Chief State School Officers 

Results for core questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



Customer Satisfaction Index 


51 


63 


s 


How satisfied are you with ED’s products and services 


51 


69 


s 


How well ED's products and services meet expectations 


51 


60 


V 


How well ED compares with ideal products and services 


50 


57 




ED Staff/Coordination 


53 


73 


s 


Knowledge of relevant legislation, regulations, policies, and procedures 


65 


85 


V 


Responsiveness to your questions 


55 


74 


V 


Accuracy of responses 


58 


80 


V 


Sufficiency of legal guidance in responses 


49 


69 


s 


Consistency of responses with ED staff from different program offices 


44 


66 


s 


Collaboration with other ED offices in providing relevant services 


41 


59 


V 


ED-funded Technical Assistance 


64 


75 


✓ 


Knowledge of relevant legislation, regulations, policies, and procedures 


70 


81 


s 


Responsiveness to your questions 


70 


80 




Accuracy of responses 


69 


77 




Sufficiency of legal guidance in responses 


59 


64 




Consistency of responses with ED staff 


56 


72 


s 


Collaboration with ED staff in providing relevant services 


60 


75 


s 


Collaboration with other ED-funded providers of technical assistance 


61 


74 


s 


Online Resources 


54 


72 


s 


Ease of finding materials online 


50 


65 


s 


Ease of submitting information to ED via the web 


55 


76 


s 


Technology 


56 


72 


s 


ED’s effectiveness in using technology to deliver its services 


59 


71 


s 


ED's quality of assistance 


55 


73 


s 


Effectiveness of automated process in improving state’s reporting 


56 


68 


s 


Expected reduction in federal paperwork 


44 


75 


V 


Documents 


51 


69 


✓ 


Clarity 


49 


66 


s 


Organization of information 


50 


72 


V 


Sufficiency of detail to meet your program needs 


48 


68 


V 


Relevance to your areas of need 


54 


71 


V 


Comprehensiveness in addressing the scope of issues that you face 


52 


67 


s 



Complaint 


21% 


11% 


Issued a formal complaint about assistance received from ED staff member 


21% 


11% 



Sample Size 19 18 



2008 



55 



CFI 



Group 



Cloes Fomell International 




Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OELA - Title III State Directors 

Results for core questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



Customer Satisfaction Index 


53 


59 


s 


How satisfied are you with ED's products and services 


59 


63 




How well ED's products and services meet expectations 


52 


59 


s 


How well ED compares with ideal products and services 


48 


56 


s 


ED Staff/Coordination 


66 


75 


s 


Knowledge of relevant legislation, regulations, policies, and procedures 


74 


82 


s 


Responsiveness to your questions 


74 


83 


V 


Accuracy of responses 


68 


79 


V 


Sufficiency of legal guidance in responses 


59 


71 


V 


Consistency of responses with ED staff from different program offices 


52 


64 


V 


Collaboration with other ED offices in providing relevant services 


60 


63 




ED-funded Technical Assistance 


78 


75 




Knowledge of relevant legislation, regulations, policies, and procedures 


80 


79 




Responsiveness to your questions 


79 


82 




Accuracy of responses 


80 


80 




Sufficiency of legal guidance in responses 


74 


69 




Consistency of responses with ED staff 


75 


73 




Collaboration with ED staff in providing relevant services 


78 


76 




Collaboration with other ED-funded providers of technical assistance 


79 


70 


V 


Online Resources 


61 


67 


V 


Ease of finding materials online 


58 


61 




Ease of submitting information to ED via the web 


64 


71 


s 


Technology 


62 


68 


✓ 


ED’s effectiveness in using technology to deliver its services 


61 


70 


y 


ED’s quality of assistance 


63 


69 


v 


Effectiveness of automated process in improving state’s reporting 


64 


69 


V 


Expected reduction in federal paperwork 


58 


62 




Documents 


58 


66 


✓ 


Clarity 


60 


65 




Organization of information 


64 


69 


s 


Sufficiency of detail to meet your program needs 


52 


63 


s 


Relevance to your areas of need 


66 


73 


y 


Comprehensiveness in addressing the scope of issues that you face 


48 


58 


s 


Complaint 


0% 


3% 


•/ 


Issued a formal complaint about assistance received from ED staff member 


0% 


3% 





Sample Size 44 34 



2008 



56 



CFI 



Group 



Cloes Fomell International 




Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OELA - Title III State Directors 

Results for custom questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



OELA - Title III State Directors 


66 


68 


Number of Respondents 


(n=44) 


(n=33) 


Providing you an interpretation of Title III (Program Officers) 


67 


78 


Helping you with your implementation of Title III in your state (Program Officers) 


67 


74 


Providing you with the information you needed (Web site) 


70 


73 


Helping you inform programs serving ELLs in your state (Web site) 


68 


73 


Effectiveness of relationship between Title III and Title 1 in encouraging collaboration 


53 


46 


Helping familiarize you with the Biennial Report form 


64 


65 


Allowing for your input and comments for refining the Biennial Report form 


59 


65 



Sample Size 44 34 



2008 



57 



CFI 



Group 



Claes Fornell International 





Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OESE - State Educational Technology Directors 

Results for core questions compared to 2007 scores 





2007 Scores 


2008 Scores Significant Difference 


Customer Satisfaction Index 


59 


62 


How satisfied are you with ED's products and services 


67 


68 


How well ED's products and services meet expectations 


58 


60 


How well ED compares with ideal products and services 


54 


57 


ED Staff/Coordination 


78 


81 


Knowledge of relevant legislation, regulations, policies, and procedures 


84 


82 


Responsiveness to your questions 


80 


82 


Accuracy of responses 


79 


84 ✓ 


Sufficiency of legal guidance in responses 


81 


82 


Consistency of responses with ED staff from different program offices 


74 


80 ✓ 


Collaboration with other ED offices in providing relevant services 


67 


72 


ED-funded Technical Assistance 


82 


83 


Knowledge of relevant legislation, regulations, policies, and procedures 


83 


86 


Responsiveness to your questions 


86 


87 


Accuracy of responses 


86 


85 


Sufficiency of legal guidance in responses 


79 


84 


Consistency of responses with ED staff 


83 


87 


Collaboration with ED staff in providing relevant services 


84 


81 


Collaboration with other ED-funded providers of technical assistance 


81 


80 


Online Resources 


59 


61 


Ease of finding materials online 


57 


61 


Ease of submitting information to ED via the web 


64 


64 



Technology 


64 


65 






ED’s effectiveness in using technology to deliver its services 


69 


67 






ED’s quality of assistance 


62 


70 






Effectiveness of automated process in improving state’s reporting 


60 


66 






Expected reduction in federal paperwork 


57 


64 






Documents 


69 


73 


✓ 




Clarity 


69 


72 






Organization of information 


70 


74 






Sufficiency of detail to meet your program needs 


68 


72 






Relevance to your areas of need 


73 


78 


s 




Comprehensiveness in addressing the scope of issues that you face 


63 


70 


s 




Complaint 


3% 


0% 


•/ 




Issued a formal complaint about assistance received from ED staff member 


2% 


0% 







Sample Size 



40 41 



2008 



58 



CFI 



Group 



Claes Fornell International 




Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OESE - State Educational Technology Directors 

Results for custom questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



OESE - State Educational Technology Directors 


74 


79 


Number of Respondents 


(n=40) 


(n=41) 


Providing you an interpretation of Title II, Part D 


79 


82 


Helping you with your implementation of Title II, Part D 


74 


82 


Usefulness of guidance document provided by the EETT program office 


74 


79 


Usefulness of the information presented at SETDA meetings 


68 


70 


Helping you with your compliance efforts 


71 


78 


Helping you to improve performance results 


66 


73 


Effectiveness of relationship with EETT program office 


82 


86 



Sample Size 40 41 



2008 



59 



CFI 



Group 



Claes Fornell International 





Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OESE - State Title I Directors 

Results for core questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



Customer Satisfaction index 


67 


63 




How satisfied are you with ED’s products and services 


72 


68 




How well ED's products and services meet expectations 


65 


61 




How well ED compares with ideal products and services 


63 


59 




ED Staff/Coordination 


77 


81 


s 


Knowledge of relevant legislation, regulations, policies, and procedures 


84 


90 


s 


Responsiveness to your questions 


75 


79 




Accuracy of responses 


84 


90 


s 


Sufficiency of legal guidance in responses 


78 


80 




Consistency of responses with ED staff from different program offices 


70 


74 




Collaboration with other ED offices in providing relevant services 


63 


67 




ED-funded Technical Assistance 


72 


73 




Knowledge of relevant legislation, regulations, policies, and procedures 


69 


74 


s 


Responsiveness to your questions 


78 


76 




Accuracy of responses 


75 


76 




Sufficiency of legal guidance in responses 


64 


63 




Consistency of responses with ED staff 


71 


73 




Collaboration with ED staff in providing relevant services 


70 


72 




Collaboration with other ED-funded providers of technical assistance 


71 


73 





Online Resources 


56 


57 


Ease of finding materials online 


49 


47 


Ease of submitting information to ED via the web 


66 


67 


Technology 


64 


64 


ED’s effectiveness in using technology to deliver its services 


66 


67 


ED's quality of assistance 


63 


62 


Effectiveness of automated process in improving state's reporting 


64 


63 


Expected reduction in federal paperwork 


59 


57 


Documents 


74 


74 


Clarity 


76 


76 


Organization of information 


76 


75 


Sufficiency of detail to meet your program needs 


69 


71 


Relevance to your areas of need 


78 


79 


Comprehensiveness in addressing the scope of issues that you face 


69 


69 


Complaint 


0% 


0% 


Issued a formal complaint about assistance received from ED staff member 


0% 


0% 



Sample Size 48 34 



2008 



60 



CFI 



Group 



Claes Fornell International 




Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OESE - State Title I Directors 

Results for custom questions compared to 2007 scores 





2007 Scores 


2008 Scores Significant Difference 


OESE - State Title 1 Directors 


72 


78 


Number of Respondents 


(n=48) 


(n=34) 


Usefulness of the training offered through the Enhancing Program Performance Contract 


78 


82 


Usefulness of technical assistance on Neglected and Delinquent 


73 


74 


Usefulness of technical assistance on Even Start 


74 


80 


Usefulness of technical assistance on Homeless Education 


77 


80 


Availability of information on monitoring for Title 1 


77 


80 


Usefulness of information on monitoring for Title 1 


77 


79 


SASA's effectiveness in using technology to provide information 


67 


74 


SASA’s effectiveness in using technology to enhance communication between ED and the state 


65 


75 S 


Sample Size 


48 


34 



2008 



61 



CFI 



Group 



Claes Fornell International 





Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OESE - State Title V, Part A Directors 

Results for core questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



Customer Satisfaction index 


73 


69 


How satisfied are you with ED’s products and services 


79 


74 


How well ED's products and services meet expectations 


71 


64 


How well ED compares with ideal products and services 


69 


66 


ED Staff/Coordination 


83 


85 


Knowledge of relevant legislation, regulations, policies, and procedures 


85 


87 


Responsiveness to your questions 


83 


84 


Accuracy of responses 


86 


89 


Sufficiency of legal guidance in responses 


84 


88 


Consistency of responses with ED staff from different program offices 


79 


82 


Collaboration with other ED offices in providing relevant services 


84 


81 


ED-funded Technical Assistance 


77 


83 


Knowledge of relevant legislation, regulations, policies, and procedures 


76 


80 


Responsiveness to your questions 


80 


87 


Accuracy of responses 


80 


83 


Sufficiency of legal guidance in responses 


73 


89 


Consistency of responses with ED staff 


76 


89 


Collaboration with ED staff in providing relevant services 


75 


78 


Collaboration with other ED-funded providers of technical assistance 


81 


80 


Online Resources 


68 


63 


Ease of finding materials online 


63 


57 


Ease of submitting information to ED via the web 


73 


79 


Technology 


74 


69 


ED's effectiveness in using technology to deliver its services 


82 


75 


ED’s quality of assistance 


75 


75 


Effectiveness of automated process in improving state’s reporting 


73 


74 


Expected reduction in federal paperwork 


62 


51 



Documents 


81 


77 


Clarity 


83 


77 


Organization of information 


84 


79 


Sufficiency of detail to meet your program needs 


80 


73 


Relevance to your areas of need 


83 


80 


Comprehensiveness in addressing the scope of issues that you face 


78 


77 



Complaint 


0% 


0% 


Issued a formal complaint about assistance received from ED staff member 


0% 


0% 



Sample Size 21 20 



2008 



62 



CFI 



Group 



Cloes Fomell International 




Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OESE - State Title V, Part A Directors 

Results for custom questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



OESE - State Title V, Part A Directors 


85 


82 


Number of Respondents 


(n=20) 


(n=20) 


Providing you an interpretation of Title V, Part A 


87 


85 


Helping you with your implementation of Title V, Part A 


88 


80 


Usefulness of the guidance document 


86 


83 


Usefulness of the information presented at national meetings 


87 


85 


Helping you with your compliance efforts 


78 


69 


Helping you to improve performance results 


76 


64 


Effectiveness of relationship with Title V, Part A program office 


88 


84 



Sample Size 21 20 



2008 



63 



CFI 



Group 



Claes Fornell International 





Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OSERS/OSEP- Lead Agency Early Intervention Coordinators 

Results for core questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



Customer Satisfaction index 


64 


64 




How satisfied are you with ED’s products and services 


68 


70 




How well ED's products and services meet expectations 


62 


64 




How well ED compares with ideal products and services 


60 


57 




ED Staff/Coordination 


78 


80 




Knowledge of relevant legislation, regulations, policies, and procedures 


79 


85 


s 


Responsiveness to your questions 


78 


82 




Accuracy of responses 


80 


81 




Sufficiency of legal guidance in responses 


77 


74 




Consistency of responses with ED staff from different program offices 


69 


78 


V 


Collaboration with other ED offices in providing relevant services 


74 


79 




ED-funded Technical Assistance 


80 


86 


V 


Knowledge of relevant legislation, regulations, policies, and procedures 


83 


87 


s 


Responsiveness to your questions 


85 


92 


V 


Accuracy of responses 


83 


89 


y 


Sufficiency of legal guidance in responses 


71 


69 




Consistency of responses with ED staff 


77 


85 


s 


Collaboration with ED staff in providing relevant services 


81 


87 


s 


Collaboration with other ED-funded providers of technical assistance 


81 


89 


V 



Online Resources 


72 


73 




Ease of finding materials online 


69 


63 




Ease of submitting information to ED via the web 


76 


83 




Technology 


66 


64 




ED’s effectiveness in using technology to deliver its services 


73 


77 




ED's quality of assistance 


66 


63 




Effectiveness of automated process in improving state's reporting 


65 


63 




Expected reduction in federal paperwork 


47 


39 




Documents 


69 


70 




Clarity 


69 


70 




Organization of information 


71 


72 




Sufficiency of detail to meet your program needs 


66 


67 




Relevance to your areas of need 


73 


78 




Comprehensiveness in addressing the scope of issues that you face 


65 


65 




Complaint 


3% 


3% 




Issued a formal complaint about assistance received from ED staff member 


3% 


3% 





Sample Size 29 36 



2008 



64 



CFI 



Group 



Claes Fornell International 




Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OSERS/OSEP- Lead Agency Early Intervention Coordinators 

Results for custom questions compared to 2007 scores 





2007 Scores 


2008 Scores Significant Difference 


OSERS - Lead Agency Early Intervention Coordinates 


78 


83 


Number of Respondents 


(n=28) 


(n=36) 


Staff responsiveness to answering questions 


79 


85 


Supportiveness in helping you complete your state's federally required performance plans 


81 


85 


Dissemination of accurate information 


79 


83 


Dissemination of information in a timely manner 


67 


78 


Centers' responsiveness to answering questions 


82 


00 

CD 

\ 


Centers' responsiveness to information requests 


81 




Impact on State's SPP improvement targets 


- 


84 


Effectiveness in addressing systems improvement issues of the state 


57 


61 


Sample Size 


29 


36 



2008 



66 



CFI 



Group 



Claes Fornell International 





Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OSERS/OSEP- State Directors of Special Education 

Results for core questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



Customer Satisfaction Index 


63 


61 




How satisfied are you with ED's products and services 


69 


67 




How well ED's products and services meet expectations 


61 


59 




How well ED compares with ideal products and services 


58 


54 




ED Staff/Coordination 


75 


65 


y 


Knowledge of relevant legislation, regulations, policies, and procedures 


84 


80 




Responsiveness to your questions 


73 


64 


V 


Accuracy of responses 


81 


70 


s 


Sufficiency of legal guidance in responses 


72 


58 


s 


Consistency of responses with ED staff from different program offices 


66 


62 




Collaboration with other ED offices in providing relevant services 


62 


51 


s 


ED-funded Technical Assistance 


74 


72 




Knowledge of relevant legislation, regulations, policies, and procedures 


77 


76 




Responsiveness to your questions 


80 


79 




Accuracy of responses 


76 


74 




Sufficiency of legal guidance in responses 


71 


63 


s 


Consistency of responses with ED staff 


73 


71 




Collaboration with ED staff in providing relevant services 


75 


73 




Collaboration with other ED-funded providers of technical assistance 


68 


67 





Online Resources 


71 


71 




Ease of finding materials online 


61 


65 




Ease of submitting information to ED via the web 


82 


77 




Technology 


61 


58 




ED’s effectiveness in using technology to deliver its services 


73 


66 


s 


ED’s quality of assistance 


62 


57 




Effectiveness of automated process in improving state’s reporting 


59 


57 




Expected reduction in federal paperwork 


48 


52 




Documents 


69 


71 




Clarity 


68 


70 




Organization of information 


72 


76 




Sufficiency of detail to meet your program needs 


66 


67 




Relevance to your areas of need 


74 


80 


s 


Comprehensiveness in addressing the scope of issues that you face 


63 


64 




Complaint 


0% 


6% 


•/ 


Issued a formal complaint about assistance received from ED staff member 


0% 


6% 





Sample Size 38 34 



2008 



66 



CFI 



Group 



Claes Fornell International 




Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OSERS/OSEP- State Directors of Special Education 

Results for custom questions compared to 2007 scores 





2007 Scores 


2008 Scores Significant Difference 


OSERS/OSEP - State Directors of Special Education 
Number of Respondents 


71 

(n=37) 


75 

(n=34) 


Staff responsiveness to answering questions 


75 


75 


Supportiveness in helping you complete your state’s federally required performance plans 


74 


77 


Dissemination of accurate information 


72 


75 


Dissemination of information in a timely manner 


58 


70 <4 


Centers' responsiveness to answering questions 


74 


81 


Centers' responsiveness to information requests 


77 


81 


Impact on State's SPP improvement targets 


- 


72 


Effectiveness in addressing systems improvement issues of the state 


63 


63 


Sample Size 


38 


34 



2008 



67 



CFI 



Group 



Claes Fornell International 





Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OPEPD - EDEN/ED Facts Coordinators 

Results for core questions compared to 2007 scores 





2007 Scores 


2008 Scores 


Significant Difference 


Customer Satisfaction Index 


62 


69 


s 


How satisfied are you with ED's products and services 


67 


75 


s 


How well ED's products and services meet expectations 


57 


68 


V 


How well ED compares with ideal products and services 


59 


62 




ED Staff/Coordination 


78 


80 




Knowledge of relevant legislation, regulations, policies, and procedures 


82 


84 




Responsiveness to your questions 


83 


85 




Accuracy of responses 


83 


84 




Sufficiency of legal guidance in responses 


80 


80 




Consistency of responses with ED staff from different program offices 


71 


69 




Collaboration with other ED offices in providing relevant services 


70 


73 




ED-funded Technical Assistance 


76 


87 


✓ 


Knowledge of relevant legislation, regulations, policies, and procedures 


74 


83 


✓ 


Responsiveness to your questions 


81 


92 


✓ 


Accuracy of responses 


78 


89 


V 


Sufficiency of legal guidance in responses 


68 


81 


V 


Consistency of responses with ED staff 


75 


87 


y 


Collaboration with ED staff in providing relevant services 


79 


90 


s 


Collaboration with other ED-funded providers of technical assistance 


71 


87 


V 


Online Resources 


67 


72 


✓ 


Ease of finding materials online 


67 


71 




Ease of submitting information to ED via the web 


67 


74 


s 


Technology 


70 


73 




ED’s effectiveness in using technology to deliver its services 


76 


75 




ED’s quality of assistance 


74 


79 


s 


Effectiveness of automated process in improving state’s reporting 


60 


72 


V 


Expected reduction in federal paperwork 


67 


67 




Documents 


69 


74 


✓ 


Clarity 


68 


72 




Organization of information 


71 


76 


s 


Sufficiency of detail to meet your program needs 


67 


72 


V 


Relevance to your areas of need 


73 


80 


V 


Comprehensiveness in addressing the scope of issues that you face 


64 


72 


s 



Complaint 


3% 


0% 


Issued a formal complaint about assistance received from ED staff member 


3% 


0% 



Sample Size 



30 33 



2008 



68 



CFI 



Group 



Claes Fornell International 




Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OPEPD - EDEN/ED Facts Coordinators 

Results for custom questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



OPEPD - EDEN/EDFacfs Coordinators 


82 


84 




Number of Respondents 


(n=30) 


(n=33) 




Timeliness of the support 


90 


90 




Usefulness of the support 


88 


90 




Accuracy of information 


85 


88 




Relevance of the support 


88 


90 




Federal mandates for data collection and submission 


65 


75 




Streamline Federal data collection and submission processes 


67 


71 




Improve State data collection and submission processes 


62 


73 




Expected reduction in federal paperwork because of the EDEN data submission process 


63 


67 




Expected improvement in education performance measurement 


65 


71 




Training provided by the EDEN/EDFacfs team 


76 


83 




Department's Partner Support Center 


94 


94 




Sample Size 


30 


33 





2008 



69 



CFI 



Group 



Claes Fornell International 





Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OVAE - Career and Technical Education State Directors 

Results for core questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



Customer Satisfaction Index 


62 


63 




How satisfied are you with ED’s products and services 


67 


68 




How well ED's products and services meet expectations 


62 


63 




How well ED compares with ideal products and services 


56 


56 




ED Staff/Coordination 


76 


77 




Knowledge of relevant legislation, regulations, policies, and procedures 


81 


85 




Responsiveness to your questions 


78 


77 




Accuracy of responses 


80 


79 




Sufficiency of legal guidance in responses 


70 


72 




Consistency of responses with ED staff from different program offices 


75 


74 




Collaboration with other ED offices in providing relevant services 


69 


71 




ED-funded Technical Assistance 


82 


79 




Knowledge of relevant legislation, regulations, policies, and procedures 


81 


80 




Responsiveness to your questions 


84 


81 




Accuracy of responses 


81 


80 




Sufficiency of legal guidance in responses 


76 


66 




Consistency of responses with ED staff 


79 


75 




Collaboration with ED staff in providing relevant services 


85 


83 




Collaboration with other ED-funded providers of technical assistance 


85 


79 




Online Resources 


69 


63 


✓ 


Ease of finding materials online 


64 


54 




Ease of submitting information to ED via the web 


76 


70 





Technology 


67 


66 




ED’s effectiveness in using technology to deliver its services 


73 


74 




ED's quality of assistance 


68 


66 




Effectiveness of automated process in improving state’s reporting 


67 


60 




Expected reduction in federal paperwork 


49 


56 




Documents 


69 


76 


✓ 


Clarity 


68 


75 




Organization of information 


75 


79 




Sufficiency of detail to meet your program needs 


65 


75 




Relevance to your areas of need 


75 


81 




Comprehensiveness in addressing the scope of issues that you face 


63 


69 


S 


Complaint 


2% 


0% 


V 


Issued a formal complaint about assistance received from ED staff member 


2% 


0% 





Sample Size 41 28 



2008 



70 



CFI 



Group 



Claes Fornell International 




Department of Education Office of the Chief Financial Officer 

Grantee Satisfaction Survey 



Final Report 



OVAE - Career and Technical Education State Directors 

Results for custom questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



OVAE - Career and Technical State Directors 


72 


71 


Number of Respondents 


( n =41) 


(n=28) 


User-friendliness 


76 


75 


Compatibility with state reporting systems 


68 


62 


Identifying and correcting compliance issues in your state 


77 


80 


Helping you to improve program quality 


73 


84 


Effectiveness of help improving quality of programs and accountability systems 


- 


73 


Effectiveness in keeping you informed about key issues 


- 


72 


PCRN’s usefulness to your program 


70 


71 


User-friendliness 


70 


69 


Compatibility with state reporting systems 


60 


63 



Sample Size 41 28 
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OVAE - Directors of Adult Education and Literacy 

Results for core questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



Customer Satisfaction Index 


70 


73 




How satisfied are you with ED’s products and services 


74 


76 




How well ED's products and services meet expectations 


69 


71 




How well ED compares with ideal products and services 


67 


69 




ED Staff/Coordination 


80 


83 




Knowledge of relevant legislation, regulations, policies, and procedures 


85 


87 




Responsiveness to your questions 


82 


86 




Accuracy of responses 


84 


84 




Sufficiency of legal guidance in responses 


77 


79 




Consistency of responses with ED staff from different program offices 


77 


76 




Collaboration with other ED offices in providing relevant services 


77 


81 





ED-funded Technical Assistance 


83 


86 




Knowledge of relevant legislation, regulations, policies, and procedures 


83 


86 




Responsiveness to your questions 


85 


87 




Accuracy of responses 


85 


88 




Sufficiency of legal guidance in responses 


81 


81 




Consistency of responses with ED staff 


82 


85 




Collaboration with ED staff in providing relevant services 


84 


90 




Collaboration with other ED-funded providers of technical assistance 


83 


86 




Online Resources 


67 


72 


✓ 


Ease of finding materials online 


61 


63 




Ease of submitting information to ED via the web 


73 


81 




Technology 


70 


75 


•/ 


ED’s effectiveness in using technology to deliver its services 


74 


76 




ED’s quality of assistance 


69 


79 




Effectiveness of automated process in improving state's reporting 


71 


75 




Expected reduction in federal paperwork 


62 


67 




Documents 


75 


79 


V 


Clarity 


75 


81 


s 


Organization of information 


78 


82 




Sufficiency of detail to meet your program needs 


70 


79 




Relevance to your areas of need 


79 


80 




Comprehensiveness in addressing the scope of issues that you face 


71 


76 




Complaint 


2% 


0% 


V 


Issued a formal complaint about assistance received from ED staff member 


2% 


0% 





Sample Size 47 44 
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OVAE - Directors of Adult Education and Literacy 

Results for custom questions compared to 2007 scores 



2007 Scores 2008 Scores Significant Difference 



OVAE - Directors of Adult Education and Literacy 


79 


84 


Number of Respondents 


( n =47) 


(n=44) 


Ease of reporting using the NRS web-based system 


78 


83 


Usefulness of the training offered by OVAE through its contract to support NRS 


84 


87 


Being well-organized 


70 


78 


Providing pre-planning adequate guidance 


68 


80 


Setting expectations for the visit 


70 


77 


Using state peer reviewers in the federal monitoring process 


71 


80 


Relevance of information 


85 


87 


Being up to date 


87 


88 


Usefulness to your program 


83 


86 


Usefulness of products helping your state meet AEFLA program priorities 


76 


80 


Technical assistance provided addresses your program priorities and needs 


71 


75 



Sample Size 47 44 
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The comments reported in this section appear in the original verbatim wording and format as provided by 
the respondent. 

Chief State School Officers 



Q9. Please identify your state’s best example of collaboration across offices that you would offer 
as a model for ED. 

Our agency is developing Response to Intervention models collaborative between the Office of School 
Improvement and the Office of Special Education; the curriculum unit in the Office of School Improvement 
worked with the Office of Career and Technical Education to crosswalk academic and CTE standards; the 
Educational Technology office is coordinating data collection across all office and with the Center for 
Educational Performance and Information to ensure consistent data reporting and usage. 

Regular, senior level staff meetings to help ensure collaboration. 

The Division of Educator Quality (School Leadership), the Division of Innovation and Support, and the 
Palmetto Priority Schools program (for failing schools) are working on collaborative, innovative strategies 
to turn around chronically low-performing schools. 

Q21. Please describe how ED could better use technology to deliver its services. 

First, ED needs to create services that would be of use to SEAs. 

Too many times, technical assistance sessions are held within 10 days of a grant announcement. This 
means that the cost of getting to DC (or even regional sites) skyrockets for potential applicants. Rather 
than site-based technical assistance sessions, webinars or webcasts would provide a cost-effective, 
equally information means to provide technical assistance to all potential applicants. Conference calls 
without web-based support and documents are not that helpful. 

Q35. Finally, please describe how ED can improve its service to you. 

Consistency 

BIE has a great working relationship with the current administrative staff and leadership at ED. Naturally, 
this changes as personalities change in various positions. Consistency in leadership and the 
management team at ED would ensure that the positive relationship and partnership we currently enjoy 
would continue on. Current leadership in BIE realizes the tremendous responsibility that ED has in 
providing quality education to all socio-economic and ethnic population in the United States. BIE 
appreciates that ED is our partner in addressing this quality delivery of services to Indian Children. 



Communication 

Learn what SEAs and LEAs really do. 

Less legalese and more every day language/discussion regarding products and services. 

Written communication could be clearer. ED's application of accountability approvals to states must be 
more transparent and even-handed. ED's understanding of the administration of federal programs, 
particularly in urban school districts, should derive more from experience, insight, and research and less 
from politically-based need for immediate change. 



Collaboration 

More collaboration between IDEA and NCLB on any level. 
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Clarity of Requirements/Transparency 

More technical assistance and more transparency in how decisions are made. 

Most resources tend to be highly legal, complex, and a reiteration of the laws and/or regulations. 
Materials are of a high level, but not as practical and useful to the field. States must and do rewrite 
information in simplified terms as well as how to turn 'law' into practice. 



Timelines 

The timeline for ED Facts may not be realistic due to data issues in the various states. You may need to 
adjust these deadlines. 



Posting Proposals 

Post two funded grants from each funding cycle on program Web sites. This would avoid the costs 
associated with FOIA requests-personnei time, copying time, mailing time. In this day and age, the way 
that FOIA requests are handled are archaic and obsolete. Since December 2007, we have been trying to 
get copies of funded proposals from four different US ED program offices. To date, only one (VPSC) has 
provided these proposals, and they arrived 5 months after the initial request. The others: I TEST ; Early 
Reading First, Educ Research on School Leadership (through IES) have not only ignored repeated 
requests, they have also not responded to inquiry from our U.S. Senators. We understand that personnel 
time is valuable-exactly the reason why the offices should just go ahead and post funded proposals. 

Other 

Frankly, I am not expecting the ED to meet all of a state's needs. Education is primarily the responsibility 
of a state and we should not expect the USDE to provide the same level of service that should be 
expected of State Departments of Education. 

I don't really know of anything that would be relevant. 
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Q9. Please identify your state’s best example of collaboration across offices that you would offer 
as a model for ED. 

At the state level , Title I and Title III 

Collaboration with other state content specialists to provide better instructional support to ELLs. 
Communication between Title I and Title III 

NCLB programs are housed together and collaboration across programs occurs on a daily basis and 
ongoing. Initiatives that involve more than one program area are done together or tools are developed 
together. 

No best example, but an example of where collaboration may be needed: offices should be able to offer 
similar examples of issues such as what constitutes supplant vs. supplement. 

Our State does an excellent job of collaborating. Recently, we have been collaborating on ELD Math and 
Science standards. Not only is the Title lll-LEP Coordinator involved, the math and science coordinators, 
Title I Director, Content Area Director, and Migrant Coordinator have been involved. 

The Oregon Department of Education includes in its T III on-site monitoring the ODE's Civil Rights 
Specialist. His role is to review services for ELLs from the students' rights perspective. 

We work collaboratively among the Title I, IIA, IID, III, IV, and V offices-meeting twice each month. Our 
applications are consolidated and our reviews of school districts are also consolidated. 



Q21. Please describe how ED could better use technology to deliver its services. 

It would be nice to have some Director's meetings via webinars, etc. 

More offerings via technology, such as podcasts, conference meetings. 

Use more online PD resources to provide more training and guidance to SEAs. Facilitate collaboration 
among states using online resources. 

Use Webinars 

Web-based training would be beneficial-especially in light of the expenses for traveling 



Q3 (OELA). What can OELA do over the next year to meet your state's technical assistance and 
program improvement needs? 

Be clear; be consistent. 

Be consistent in the guidance it provides; be more responsive to written requests for clarification and 
issue written guidance that is based on statute. 

Continue to provide outstanding program managers and reduce paper work and monitoring 

Continue to send out interpretations and guidance documents. Accompanying these with 
comprehensible, additional presentations is most helpful. 

Have written guidance as to how to implement Title III, especially for novice state staff. It is very difficult 
to implement a program based only on the legislation itself. 

Host a 2008 OELA Summit Meeting. 

I can't answer this question at this time. 

I need to know about my state's submission to OELA for a change of Annual Measurable Achievement 
Objectives (AMAOs). I am also looking forward to the guidance concerning supplanting. 

I think the one thing to give us more time on the notice of interpretation regarding title 1 1 AM AO's. Once 
that document is finalizes state will additional time and money to implement them. 

Inform all states of each state's decisions, allow for blanket application of decisions. Be uniform in treating 
states and let each state know - in writing - what has been determined allowable/acceptable. BE 
CONSISTENT. Have conviction in decisions and inform all states 
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Inform us when Till Native American Program grants are awarded to LEAs. 

More clarity in guidance/e. g., Title III consortia and private schools/home schools 
More guidance re: ELLs with special needs. 

No. I am very pleased with all the assistance I get. 

Nothing 

OELA's staff prompt and accurate responses are invaluable as we send questions to them when a local 
LEA needs an answer right away. Continuing to provide this service will continue to be much appreciated. 
Also, Title III State Directors meetings in which we can share and learn from each other are also 
invaluable. 

Please provide an official document, which states can use as a guidance document for Title III. 

POINT THE APPROPRIATE LEADERSHIP. DEFINE THE ROLL OF OELA IN THE LEP PARTNERSHIP. 
I AM WAITING FOR THE REGULATORY TO BE PUBLISHED. 

Provide a simple clarification of students to count as LEPs. 

Provide guidance and monitoring reports in a timely manner. 

Provide guidance on meeting AM AO, which has been promised for years and has not been given. 

Provide more guidance on AMAOs and assessments for ELLs. 

Provide proactive training and guidance instead of back tracking and retribution. Have better 
communication with SEAs. 

Provide specific models of recommended monitoring surveys. 

Publish relevant guidance. 

Responding in a timely manner to questions would be extremely helpful. Having efficient State Director's 
Annual Meetings with a specific and relevant agenda would also be most helpful. 

Showcase states that have highly developed Title III programs and use the funds effectively. 

We are waiting for the AMAO, RTI, and Special Education guidelines documents. 

We need guidance on ELP assessment and accountability. 

Work with us on the quality of the data requested and the venues through which it is provided. 

Q7 (OELA). Please describe how the working relationship between Title III and Title I could be 
improved. 

Build some requirements to collaborate into the legislation and/or guidance. 

I think we cannot have joint meetings and discussions. They don't communicate very often between the 
two offices of Title I and Title III. 

Improve collaboration at federal level that trickles down to state 
Invite Title I and Title III to the same meetings. 

It appears like these two offices work in complete isolation. Perhaps having Title I representation at the 
annual Title III Director's meeting would be a good start. 

Make Title III a part of Title I. 

More communication between the two offices and agreement on guidance - sometimes guidance is 
different between the two offices. 

More state director collaboration. Shared or overlapping meetings. 

Required regional meetings. 

Since many of the Title III functions are moving to SASA, the relationship should improve. 

The level of funding for Title III should be greatly increased. Title I seems to be able to fund most of its 
mandates, whereas Title III hardly provides funding to implement all of its mandates or requirements. 

The teeth of accountability rests with Title I because of the amount of funding and the number of school 
administrative units affected. If Title I was as knowledgeable about Title III as Title III is about Title I 
perhaps English language learners would not be marginalized but would be included in ALL STUDENTS. 
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Title III targets a small fraction of school administrative units and allows for too many accountability 
features for non Title III receiving schools to ignore accountability requirements for English language 
learners. 

There IS no working relationship with Title I. 

Title I may say one thing and then Title III says it is wrong and sites us for something Title I said was 
okay. 

Title I needs to take a more active role in this relationship and show responsibility towards the needs of 
the ELL population. 

Title III staff generally have the information on how to educate language minority students which generally 
T I staffs do not. Setting expectations and linking the work of T III and I can be a first step. 

Title III staffers need to understand Title I. 



Q35. Finally, please describe how ED can improve its service to you. 

Responsiveness/Timeliness 

Assistance from Title III needs to be clear, consistent, and timely. Guidance is never provided in writing, 
always verbal. States cannot act on verbal interpretations of statute and it changes from meeting to 
meeting. Program officer keeps changing and that results in lack of consistency in assistance. 

Be expedient with answers 

Meet projected timelines for reports and responses. 

Provide timely guidance on issues we face... 

Respond to the formal complaint! 



Communication 

Ensure staff working at the federal level is well trained and well informed regarding the policies they are 
being required to implement and oversee. Improve communication with SEAs to ensure that the 
information provided is relevant and decisions made are not out of touch with the reality of 
implementation at the state and school district level. 

Spend more time in the field understanding how states differ. 



Data Collection 

Cut-down on the EXCESSIVE amount of data collection and reporting required. 

The more data that can be collected through the EDEN system rather that written reports from states. 



Web site 

Make the site easier to navigate, with relevant information. Sometimes things can be hard to find and it 
takes a lot of time. I just end up calling. 

Mostly my experiences on the grant side has been a little bit challenging. It has been challenging to get a 
copy of the application. The electronic part is user friendly it's just hard to get to download an application. 
We had a lot of support in getting the assistance we needed. 
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Meetings 

Do not cancel the OELA Summit 

Have regional meetings for technical assistance but provide for all expenses to be covered through ED. 
Travel for state employees, even that funded by federal funds, has been greatly curtailed in many states. 



Coordination/Collaboration 

More coordination between the different tile corporations. I am involved in two and the differences in the 
support given. 



Guidance Document 

Have written guidance. 

List the ELP tests each state uses, each state's rules for LEP identification and definition of AM A0 1 
(progress) and AMA02 (proficiency). 

Please provide to states an official guidance document or regulations for Title III. NCLB is getting close to 
reauthorization and states still do not have a Title III guidance document. This is problematic in light of 
how complicated AM AOs are. 

We need guidance on implementing requirements. Guidance that has been developed through 
consultation with experts and researchers is most helpful. ED can improve services by continuing to rely 
on experts in the field to develop guidance and technical assistance. 

When guidance to one state is developed, send it to ALL states. Do not put the burden of asking on the 
states. Do not treat each state differently, do not continue to allow one state to do X and another not. 
Consistency, fairness, and more than anything - put it in writing. Develop conviction 

When new Federal requirements are issues, like designing a RTI program for ELLs and others, we need 
immediate guidelines from Title III office. If not we are left to create this documents with what we 
understand should be done. Then months or years after Title III comes out with guidelines that sometimes 
are different then ours - creating confusion and distrust on State agencies from the field. This can be 
avoided if from the beginning we are all on the same page. 



Mailing List 

Keeping a current State Director's mailing list would be great. I've tried to get on their mailing list several 
times but I still don't get their emails and updates. 

Other 

All things considered, ED is doing well! 

Continue to provide high quality technical assistance to states. [Name Deleted] has been most helpful as 
our Title III Officer. 

No, I am very pleased with the working relation that we have with them. 
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OESE - State Educational Technology Directors 

Q9. Please identify your state’s best example of collaboration across offices that you would offer 
as a model for ED. 

Have a conference call that involves programs that cross over. Ex. Title I A and Title IID 
Our state is not good at collaborating either. 

Program Requirements for Title IID Competitive-with relation to private (non-public) school participation 

The auditing of federal programs and the Title IID helps me to get to other offices to get the help I need. 

Title IID and Title IA, 1C, IIA, III, VA, and VIA/REAP - We all work on one team in order to collaborate on 
professional development, technical assistance, monitoring, and resources provided for LEAs. 

We are working with the Center for Education Performance and Information (CEPI) on data collection and 
longitudinal data systems. It would be great for the USDOE Office to work more closely with EDEN. 

EDEN is asking for ED tech data, but the USDOE is not providing the requirement to make it part of our 
state's mandatory collection. This is causing difficulties in determining how the information is collected. 

My office is also responsible for a large portion of Title IIA statewide activity dollars that is funding a 
statewide teacher portal, but Title IIA and Title IID are separate initiatives. It would be better if the 
priorities of those federal programs were more aligned. 

We have worked very hard to link the school improvement process with our NCLB programs unit. 

Q21. Please describe how ED could better use technology to deliver its services. 

Easier access to on line resources, Webinars, on line trainings, etc. Since funds are limited any type of 
TA that can be provided via a distance with out travel would be beneficial. 

ED could use the technology such as interactive video conferencing to provide more services, 
collaboration with other Title programs (connecting state directors of the different programs to talk about 
issues including parental involvement, professional development in educational technology tied to core 
content, sharing resources across states including application forms, data collection tools/strategies, 
curriculum, and digital content, etc.) Ed could help states network across programs to leverage 
resources, learn from others' successes stories, and sharing resources developed in one state with other 
states that may not have the staffing to create great resources. 

ED needs to explore more advanced technologies such pod casting video conferencing. 

ED should be encouraged to use presentation tools such as WebEx to deliver application information and 
provide more contact with state customers. 

It is difficult to answer this because we do not receive much service from ED, through technology or 
otherwise, other than what we initiate. 

Offer it from time to time. My only interaction has been a couple of times , so we need more of it. 

Program office does not tend to communicate with states. Communication with USDOE program office 
comes through SETDA or other sources. The USDOE Ed Tech office should be providing State Ed Tech 
directors with updates and information regarding what is going on. Technology is a support to ensure this 
communication happens-it is 50+ contacts for USDOE-not unmanageable. I should not have to attend a 
national SETDA meeting to find out what is happening with the OET or OESE departments at USDOE 

Provide CVC's monthly 

They need to know how to use it. Also, the EETT program doesn't collect information electronically except 
through EDEN. 21st Century collects all annual reports electronically. 

Update capacity to support IP-based videoconferencing. Post monthly or quarterly information (via email 
or online newsletter) to provide program updates, describe/facilitate connections with other programs and 
program providers, cite proven practices, etc. 
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Q8 (EdTech). Please describe how your working relationship with EETT could be improved. 

Be honest and straightforward with us, being timely with information. 



Q9 (EdTech). What can EETT do over the next year to meet your state’s technical assistance and 
program improvement needs? 

Advocacy for continued educational technology funding! 

Assist with getting clearance at to what exactly will be needed for EDFacts. 

Bi-monthly email/newsletter of relevant issues or deadlines that are coming up. 

Clearly define 8th grade technology literacy 

CONTINUE TO WORK WITH EDEN ON REPORTING REQUIREMENTS. 

EDEN is very hard to understand - please make it less technical for those of us actually having to collect 
the information. 

EETT has done an excellent job in keeping the states abreast of the current happenings in Washington. 
An improvement to me would be more EETT sponsored national meetings were the states participates 
directly with ED staff and not through any association. 

EETT should provide ongoing communication to State Ed Tech Directors-including program updates. 
EETT should not rely on SETDA or other organizations to provide communication to State Ed Tech 
Director's, but should develop ongoing communication channels on its own. The information should be 
published and/or sent directly to the states in case states are unable to attend the in-person meetings. 
EETT office needs to work in conjunction with the OET to ensure a common vision. EETT and OET need 
to work together to develop and incorporate 21st century skills within the 'work' of these programs. 

First and foremost it must be fully funded to have any impact. If fully funded, help states explore, discuss, 
share ways of evaluating impact of formula grant program and overall EETT evaluation. 

Frequent communication and increase technical assistance available. 

Give definitive answers in writing when we reach out for assistance. 

Give examples of what you believe are adequate representations of how states can meet the 8th grade 
technology literacy requirement, the teacher technology literacy and integration expectations in EDEN. 

Given the new NETS for students, most states including mine are struggling to find/develop an effective 
tool for measuring technological literacy. Existing tools are aligned to the old standards and therefore not 
addressing the indicators that we need. Any assistance in this area would be helpful. 

I don't think any at this time. 

I think consistently communicate. 

Increase it's funding. We have had a lot of reduction in funding over the past few years abd it makes it 
hard, so restore the funding back to the initial level to where we started. 

Increase the budget. 

It would be helpful if we could tie Title I ID formula funding to reporting on 8th grade tech literacy. 

It would be nice to have a place where states can visit to gather implementation ideas or view best 
practices regarding EETT. 

It would be very helpful, now that the program had had a few years of operation, for USDOE to update the 
program guidance and provide a frequently asked questions section. 

It would help if there was more consistency and cross knowledge on data items such as 8th grade tech 
literacy. Thanks. 

Keep up keeping us informed. 
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Letting states know exactly what is needed for reporting an evaluation purposes. Possibly sending a 
cable with timelines and expected deliverables for reporting purposes. 

Monthly phone conference and/or CVC for FAQ 

More contact. Additional person-to-person contact. 

More publications regarding guidance and procedures. It seems like we see more correspondence from 
SEDTA than we do from the Department. SETDA is a nice way to get the info out however due to 
declining funds some states will be unable to attend SETDA meetings and if the Department could get 
this information out in other ways and not solely rely on SEDTA that would be beneficial. 

Not much unless they have the ability to increase these funds significantly. 

Offer more meetings and guidance about monitoring and accountability in EETT and the expectations in 
the future of the program. 

Provide additional funding. 

Provide examples from other states that will assist with program improvement. 

SETDA is a good regular source of info. As long as EETT shows up and gives the best info available 
they will be able to meet our needs. Clarify the role of EDEN within EETT that would be good. 

Share more promising practice rather than talking about the process. 

Stop providing 'politically correct' answers to program questions. Responses given at meetings are 
typically vague enough to leave us knowing little more than we did previously. 

They have been hesitant to give us information. Update their information and make it more timely. 

We, in Illinois enjoy the patience that ED has given to the new staffers. Thank you for being there and 
taking the time to ease our understanding of EETT. 

Q35. Finally, please describe how ED can improve its service to you. 



Communication 

Additional and more frequent contact. 

I answered this question in the EETT question, but a more direct involvement with training and sharing 
among states directors sponsored by ED and not through the any association. Although the associations 
have a place in informing states and representing states on EETT issues on the hill, ED should have the 
direct responsibility of having all states to meet, train, inform, and share information. 

More sharing of what other states are doing. 

They have been really good about working with us. Just more consistent communications. The 
reauthorization of funding and keeping us abreast. 

Through detailed information and timelines for reporting purposes. 



Web site 

Create a useful, easy to navigate Web site. Post information regularly. Maintain Web site so resources 
are timely. 

Make items easier to find via the search mechanism. Communicate available services. We don't really 
know what services ED provides. 
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Technical Assistance 

Again, I think if there could be more TA, guidance, etc that is provided through webinars, conference 
calls, and publications would be great. It is great that ED uses other resources to get their information out 
however I don't think that is the sole way to share information with the States. Only having one or two 
meetings a year is difficult when funding may not be available for attendees. (Not asking for more funding 
simply asking that the DE consider lack of funding when setting up trainings that require travel). 

By providing accurate and timely information and provide additional technical assistance. 



Clarity on Requirements 

ED does a very good job in responding to questions and offering clarification on most issues. There does 
seem to be confusion, however, over the reporting requirements of programs because of the different 
types of reports that are collected. If an effort would be made to consolidate that information between the 
GPA, Performance Reporting, and EDEN, so that we could understand the purpose of each, that would 
be very helpful. 



Meetings 

ED needs to call a Title I ID meeting separate (or adjacent) to the SETDA Meetings. One-hour meetings 
to barely touch upon one topic is not enough. It would be great to have states work together on strategies 
for effectively measuring technological literacy or the increased capacity of teachers to integrate 
technology into teaching and learning. Or a meeting to thoroughly discuss the findings from the 10 
ESETP grants and how they could impact the work each of us is doing in our state. 

Meetings, conference calls and or training more frequently. 

Significant advance notice of meetings is especially helpful. States are very slow to approve meetings. 
Meetings often fall during peak travel periods (late June and late November or early December). Consider 
scheduling for slower periods. 



Timeliness 

Reduce turnover of stats. Provide more timely information. Ask us more what we need. 



Guidance 

ED should follow up with states on feedback from virtual site visits; ED should provide guidance and 
interpretations that are accurate related to EETT. 

Issuing program guidance a little earlier in the implementation period. 

Just keep moving forward in guiding us through what may be ATTAIN 

Other 

Get rid of the extreme testing requirements because we just don't have the resources to do it all! We want 
to help our schools to teach their kids, not test, test, test. We need a more balanced approach. I can't 
think of anything at the moment. 

I have actually experienced an increase in support from the USDOE office. 

I have no complaint with the technology office. My only complaint is dealing with EDEN. 

I realize that EETT has been slated for elimination for the last 3 years. Does that mean that the program 
is treated as an orphan that should not be supported? It appears that way when I see the services that 
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are available through ED to other areas of K1 2 education programs. That is a difficult position to work 
from, but until the program is dead, it should be treated the same as other programs. 

Less political posturing/bias. 
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OESE - State Title I Directors 

Q9. Please identify your state’s best example of collaboration across offices that you would offer 
as a model for ED. 

A team of state staff from Title I, II, and III plus Special Education and District Improvement offices are 
collaborating on the LEA Improvement process beginning with a fact-finding on-site visit. Results are 
used to develop technical assistance and the improvement plan, including selection of the corrective 
action for those LEAs in the 3rd year of improvement. 

Consolidated NCLB application of eight formula programs 

Migrant and SASA 

Monitoring of our LEAs across all NCLB program areas in one process and one instrument. 

Regular, scheduled program meetings between program offices/directors. 



Q21. Please describe how ED could better use technology to deliver its services. 

Having conference calls/webinars for Title I folks in general 

I find the conference calls awkward. If they could do a web cast or a pod cast where I could log on to. A 
blog would be a better use of everyone’s. It would have a written format, which would be easier all the 
information would be in. 

It is very difficult to find information on your Web site. Redesign it! Improve your search engine. Get a 
users group to help you. Possibly have a portal for State and district level users that is different from 
public users. Let the user decide which portal to enter. 

Podcast or other type of delivery of information so that it can be accessed at any time 
Provide more audio and Web conferences 

Sometimes the documents are read to us on the call instead of discussion taking place. 

Q8 (Title I). Please describe how SASA could better use technology to provide information. 

On conference calls, send out information ahead of time and don't read the information to us. We can 
read it. Plan these conference calls with the input of a few of the reliable state directors so that the calls 
will actually meet our needs, e.g. the differentiated accountability conference calls just began to address 
some of the real issues by the end of the third call, the political agenda appeared to get in the way of 
clear answers. At the end of the first call give send an email to Title I directors with an email address to 
send in questions. Sort the questions into frequently asked questions that would be of value to many (and 
address these in a follow up call), and state specific questions that can be answered to the state asking 
the question. Then publish a list of the answers maybe to all questions. 

Overcome time zone differences 

Use Webcasts, etc. on a more frequent basis. Right now the only use of technology on a regular basis 
are the few emails we get from the Director of SASA-which we appreciate. 



Q10 (Title I). What can SASA do over the next year to meet your state’s technical assistance and 
program improvement needs? 

Cite my state for insufficient staff in title I and not meeting the state's maintenance of effort requirement 
where we have almost no state funded positions or initiatives. Schedule a series of Just In Time 
conferences at times of the year that coincide with what we should be doing at that time of year, e.g. PNP 
Planning guidance, Program Planning supplant issues, Program evaluation, final expenditure report 
Monitoring, etc. Give examples of best practices, and strategies to work within the state government. 
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Continue to use technology to distribute information and to give state directors a chance to participate 
more frequently in meetings without the need to travel. Continue to work with other sections to streamline 
and coordinate the data collection so that EDFacts & the CSPR are aligned and are collecting the 
appropriate data. We strongly support electronic data collection, but it's been a bumpy road. 

Continue with current practices. Title I Directors have been waiting for guidance for Targeted Assistance 
programs. [Name deleted] does a good job as director. Communication and information is good. Keep 
Title I directors in the loop as information is provided to Chiefs, assessment directors, Title II A directors, 
etc. as much of the information is directly related to Title I requirements. 

Decrease amount of time it takes to receive response from email or phone questions. 

Hold more 'WebEx' kinds of conference 

I am very satisfied with the level of assistance my state receives. The responses to our questions are 
consistent regardless of the staff called. 

Information on a more timely basis. Forget the rule making that's too late. 

More direct contact-such as teleconferences and other means of communicating information without 
having to travel. Also, I am pleased that ED is sponsoring meetings on RTI and the new math research. 
Thanks 

Provide faster response to questions and concerns. 

Provide more electronic updates and interactions on all the program areas. Develop a consistent process 
and agenda for program reviews. 

Provide more guidance 

[Names deleted] are S/AS/A staff that I work with frequently. They are extremely helpful and very 
knowledgeable. 

To upon request put responses to questions in writing. 

What there doing is ok. No comment. 

While costs on travel are high, the complexity of the issues relating to Title I services to SEAs and LEAs 
are of concern. Face-to-face meetings and networking to solve problem is still my preferred method. 
Hearing the information from the source with multiple ED staff available for specific issues is helpful. 

Work more emphatically with state's to ensure alignment of technologies being used to provide technical 
assistance and to collect program data. 

Q35. Finally, please describe how ED can improve its service to you. 

Consistency 

Consistency would be good... and someone to answer our questions in a timely manner. 

I find that the Homeless Coordinator receives conflicting information as to the use of Title I funds for 
homeless students. Please provide more clarity. Also, we need much more assistance with neglected 
and delinquent. 



Availability 

Just have staff available to answer questions by phone and e-mail. 

Timeliness 

Continue to provide timely, relevant information. 

Just information on a more timely basis 



2008 



89 



CFI 



Group 



Claes Fornell International 




Department of Education Office of the Chief Financial Officer Final Report 

Grantee Satisfaction Survey 

Loosen policy oversight of technical assistance and information that could be provided by program staff. 
Response to questions take too long because of the red tape that has to be processed before answers 
can be given. 

Provide faster responses to questions and concerns. 

Speed up the response time to emails and phone calls 

Web site 

Redesign their Web page so it's more keyword search. A keyword search that works. 



Publications 

ED limits the number of publications a state can receive, specifically the number of EDGAR copies a state 
can get because it is on the Web. However, grantees do not always have access to the Web. We should 
be able to obtain as many copies as are needed in order to share with grantees. I also think all grant 
programs should have training for new directors on a yearly basis. Training should not only include 
program requirements but should also include regulations and OMB Circulars requirements as well. 



Communications 

Having more policy memos that they put out, so they don't update them as much as they need to. I'd like 
the policy letters sent to states to available to states, I’d like there to be database of policy questions and 
answers. 

Providing models and examples to the greatest extent possible. 

Really appreciate the listservs, Webcast, Eden, etc. as well as the informal emails received directly from 
the S/AS/4 Director. 



Guidance 

Overall, the service from the SASA office of OESE has been very good. We are still waiting for new 
guidance on Title I targeted assistance programs, but other guidance recently issued on Schoolwide and 
fiscal has been helpful. Other sections of ED have been less helpful. There has been little concrete 
guidance from Title III and information has often been less than complete. The new proposed 
interpretations should be helpful in clarifying the expectations nationwide. The other large concern is the 
timing of the CSPR data collection requirements and the coordination with the EDFacts data collection. 
The CPSR data requirements are being finalized AFTER the year is over and it is too late to ask LEAs to 
collect data in a different way. At the same time, the EDFacts data collections are being finalized for the 
NEXT school year and sometimes the two data collections are still not in alignment, especially for LEP 
data and teacher quality data. We strongly support electronic data collections, but the data elements need 
to be determined BEFORE the school year starts and they must be aligned between CSPR and EDFacts. 



Other 

I am very pleased with my contacts at S/4SA / feel that there are very responsive to my needs. They are 
to be commended. 



This is a difficult one because I understand ED's limitations in terms of providing guidance that is 
responsive to the needs of any particular state. That being said, however, in too many instances the 
needs of individual state's are not addressed when in fact if ED extended itself, they may be able to do 
so. 
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OESE - State Title V, Part A Directors 

Q9. Please identify your state’s best example of collaboration across offices that you would offer 
as a model for ED. 

Interagency meeting and collaborative initiatives and projects concerning student learning. 

The technical assistant oversight team. 

Q21. Please describe how ED could better use technology to deliver its services. 

We could receive more information electronically. 

We had a 'virtual' monitoring in November of 2006. We are still waiting for the written response. 
Ridiculous... 



Q8 (Title V). Please describe how your working relationship with the Title V, Part A program office 
could be improved. 

[NO RESPONSES] 

Q9 (Title V). What can the Title V, Part A program office do over the next year to meet your state’s 
technical assistance and program improvement needs? 

Continue to clarify the implications of the 'zero' appropriation. Continue efforts to restore funding or 
discontinue the legislation 

I don't know, because there are not going to be any funds next year. Help fund the program. 

It's not going to funded next year. 

Keep us informed of developments 

Provide clear and concise information to the states. Be more accessible and responsive. 

Satisfied 

The Title V Part A program office is doing all that can be reasonably expected for a zero funded program. 
I don't know what more would be needed or prudent in the absence of refunding. 

When they perform an audit, or a monitoring visit, please make sure the written synopsis of the event is 
completed by 30-60 days. 

Work to get Federal funding back into the Title V, Part A Program by promoting the things that Title V, 
Part A provides for states and monitor how states are doing during the coming year of zero funding. 
Continue holding national meetings to assist state (SEA) program managers through this difficult time. 



Q35. Finally, please describe how ED can improve its service to you. 

Technology 

Continue to utilize technology and develop guidance incorporating questions from the field 



Responsiveness 

Be more responsive and accessible. 
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Web site 

Easier Web site to find guidance etc 
ED's Web site is very difficult to work with. 

I recognize and appreciate that ED folks work hard and try their best. I believe that they, like us, carry 
more workload than is reasonable, and this prevents more state-level support. I imagine ED staff 
turnover affects their ability to understand programming over time just as it does for us. Many of the 
resources ED provides are outdated, the Web sites are difficult to navigate, and the emphasis on 
'research/evidenced based' 'best practices' has reached absurd levels. Many of these identified 
exemplary programs have startlingly limited value across regions, cultures, communities, and with other 
subpopulations. Moreover they often require a level of fidelity and a length of implementation that defy 
the realities of today's schools. It is frustrating to guide districts and schools away from localized 
practices toward programming that will likely not accomplish what it advertised as being capable of 
accomplishing. 

The website should be improved since it is difficult to locate information by category and the search 
function is almost useless; i.e. often over 500 items result from a specific search. 



Staff 

Unless the program is funded unless they start funding. They’re not going to have staff. If they do then 
they need more staff. We are more knowledgeable than they are. 



Other 

Get the entitlement dollars out more timely. 

If every office was as responsive and as communicative as the Title I office, you'd have an amazing 
federal agency! 

Satisfied 
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OSERS/OSEP - Lead Agency Early Intervention Coordinators 

Q9. Please identify your state’s best example of collaboration across offices that you would offer 
as a model for ED. 

National Transition Initiative 

Our state uses our state interagency coordinating council and committees of the council to assure all 
agencies and many parents and providers are involved with the program and issues. There is no similar 
method at the federal level and it needs to be followed nationally to as many of the states issues are 
really federal level items that have never been aligned across funding sources. (CMS, Samsa, Title IV E, 
etc.) 

Q21. Please describe how ED could better use technology to deliver its services. 

More opportunities. If you miss the call, most of the time you just miss it. If you have a small staff and a 
busy schedule, you cannot always join in. More notice... you can't always drop everything for a call that's 
on short notice. Calls have a limited scope, so you don't always get a chance to ask your burning 
questions. 

Need one place to look on line or have them all linked together. We have to look at multiple sites, which 
is not realistic or helpful for easy references. Teleconference and materials need to be holistic and not so 
short time related. OSEP needs to make realistic expectations as need requirements, data points, and 
processes are added. We spent time on month phone call that have different or ongoing updates to 
materials which meant inches of paper and that many changes. Figure out a process and stay with it. To 
do so work with state for realities of work load and what can be done within the current funding levels. 
Part C is monitored as heavily as part b. However, Part B has much higher funding levels. There needs 
to be a balance in process expectations and clear and concise directions, and materials. Information 
needs to be received by states with enough lead-time to meet the requests. This does not currently 
happen and states are in constant run mode to try to meet the OSEP expectations. We do not see the 
outcomes for children and families being improved but rather must shift to provide OSEP with constant 
and changing demands. Put balance into service delivery and monitoring activities. 

Use of go-to-meeting or other Web conferencing technology to enhance conf. calls 

Q9 (OSERS). What can OSEP do over the next year to meet your state’s technical assistance and 
program improvement needs? 

1. Consolidate the requirements so that they are not scattered thru TA call notes, correspondence, and 
other guidance materials. 2. Decrease the reporting requirements, they are out of the scope of what a 
government administrative office can manage and are bordering on research projects. States which do 
not have the resources required to manage them are floundering. 3. Accept a state's self-assessment 
of its deficits, without forcing sanctions and penalties to local programs, which may be out of the local 
program's control. 

Better and more widely available TA around general supervision and monitoring. This could be a 
combination of on-line, site visits, conferences. What's currently available is not enough given the 
emphasis on accountability. 

Continue the good work! 

Continue to provide timely responses to questions 

Finalize Part C regulations. Seek increased appropriations for Part C. Stabilize interpretations and stop 
changes in accountability reporting. Fund data system development. 

Get more state contacts. With the SPP/APFt process being so cumbersome and time-consuming, state 
contacts just cannot handle the increased workload effectively. 

I generally find those responsive to helping me with any issues. 
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Implement a one-year moratorium on annual reporting in order to allow states to 'catch up' with what 
OSEP expects! Implementing changes to State Plans does not occur overnight and with personnel 
turnover a critical factor, time is needed. 

Increase use of technology to the disseminate information such as Webinar and Web meetings. They 
should schedule conference back to back so we don't have to fly so much for example data and 
accountability meetings. 

Issue a letter stating that the Baltimore meeting and the December Washington meeting are required for 
Part C Coordinators. Our states' department directors are getting very tight on approving travel to 
national conferences and it is critical that we be there to learn from each other and meet with OSERS 
staff and technical assistance staff. Without the requirement, I worry that fewer and fewer of the 
coordinators will get approval to attend and will miss out on these extremely important meetings. 

It is very difficult to attend all the teleconferences-it keeps me from getting work accomplished as 
sometimes they are not relevant to my needs 

Our RRC is slowly developing more interest and expertise in Part C (it took a very long time). And, 
although they don't have anyone on staff, they are very good about seeking out expertise from other TA 
providers or other individuals to help us. I think this is probably the way to go. 

Provide more national publications/training materials that are directed to the Part C systems 

Publish Part5 C Regs 

Put the funding back to TA systems to meet service deliver issues and needs and such high degree of TA 
funding on OSEP monitoring tasks. If it takes all the TA then OSEP should see that the process is not 
realistic. States do not have more resources and can't shift the funding to write reports. Therefore the on 
going TA for the service provides that used to be provided has shifted to monitoring activities. This is not 
supporting that state and local program partnerships needed to keep up with family, parent, and children 
needs or resources. 

Relax a little and let the work happen; access more resources; fund data systems work 

Strengthen NECTAC and stop pushing the RRC to provide TA in early childhood, which they do not 
specialize in. 

They are providing all kinds of training so nothing at this point. 

Timely distribution of information to accurately address APR indicators more timely. 

Visit the state to learn the states programs and circumstances. 

Whatever OSEP can do to facilitate the development of Part C federal regulations would be greatly 
appreciated. 

When providing TA over Webinars, teleconference calls, etc. provide follow-up in writing in a timely 
manner. If states cannot participate in one of these TA events than critical information is missed. 

Q35. Finally, please describe how ED can improve its service to you. 

Communication 

Bring states, families, and SICC chairs together to assure accountability and data collection. Reduce 
paperwork, as we cannot keep this workload up. Work with congress through DOE secretary and others 
in high chain of command for real funding and resources needed for EIS. The cost saving for all needs to 
be used to demonstrate why funding increases are critical. Stop changing things each month, quarter or 
year. Obtain consistency. Write clear and concise rules and regulations. If is takes OSEP attorneys to 
write them and then interpret them, what does that mean for families and states who are trying to use 
them??? The massage should be clear that too much regulation becomes as ineffective as not enough or 
none. Recognize the urgency and take reduction action now. 

Come and learn about the states program. 

Sometimes its responses are 'legalese' and don't really answer a question-be more specific 
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The correspondence needs to be more concise and clearer. I think the ED staff try to be clear but the ED 
legal staff get hold of it and make it less clear. The Part C regs have not been finalized 3 1/2 after the 
IDEA statute was passed. This causes confusion as we have a new statute but old and draft regs. They 
need to get the regs out ASAP 



Mentoring 

I am new to the position and it just seems foreign to me-l would have benefited from a mentor- 



Reporting Burden 

I understand the legal review process is necessary but it makes all relations and communications stilted 
and often not very practical. Lighten up. 

Reduce the burden of increasing requirements. Finalize the Part C regulations. 



Technical Assistance 

More specific focus for Part C education-lead states 
More targeted TA around general supervision. 

Timeliness 

Provide more timely feedback and uniformly apply interpretations and implementation of regulations 
across states. 



Web site 

I find the search feature of the Website difficult to use. Perhaps organizing it more like the NECTAC site 
would make it more user-friendly. 

Well I’d like to see a few more tweaks in the portals would be great. They are getting better and much 
more user-friendly. 



Other 

Since we are not a Dept, of Education, we don't actually see too many products from ED related to infants 
and toddlers since the focus is usually NCLB or Part B. So, my ratings reflect mostly a lack of services or 
products related to Part C. Do we have Part C regs yet? No. But a second set of Part B regs are on 
their way to OMB. 
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OSERS/OSEP - State Directors of Special Education 

Q9. Please identify your state’s best example of collaboration across offices that you would offer 
as a model for ED. 

I can't think of one. They are continually asked to collaborate but it appears that they do not take the time 
to get to know each other’s requirements. I have sat in more than one meeting when they have been told 
that things do not align with NCLB to be ignored. 

NCLB (AYP) and Voc Rehab (transition) 

OSEP office and their legal 

Project based cross division teams 

Response to Intervention team contains members from Special Education, Curriculum, Reading First, 
Early Childhood, and Bilingual Education. 

Rtl brings a number of our divisions together to explore ways in which we can work toward ensuring 
students are accessing appropriate services and best practices are available to all schools. We brought 
together representatives of a wide swath of educational professions to set a direction for us to move in 
implementing Rtl at all levels. 

Some work being done between OSEP and RSA on summaries of performance for transitioning students. 

Special education, migrant, Title 1, and ELL collaborating to provide guidance on meeting the needs of 
culturally or linguistically diverse students without inappropriately identifying them as students with 
disabilities. 

The Division of Exceptional Children Services and Division of Federal Programs (Title 1, etc.), attempt to 
conduct joint monitoring of local school districts. We have developed a cross-walk to identify 
commonalities to prevent duplication and to maximize our limited state level resources. 

The partnership that they between IDA OESE Title I. 

Title 1 and IDEA are regular collaborators in our office. 

We have a monthly NCLB team meeting that includes special education, Title, state assessment, data 
collection, HQT, and the secretary and/or deputy secretary of education. We discuss all relevant topics 
related to NCLB and IDEA. 

Work on our SPDG (creation and implementation of an integrated three-tiered model of intervention, 
known as the MIM). 

Q21. Please describe how ED could better use technology to deliver its services. 

Calls are only offered at one time and not staggered and in our state we cannot access because of state 
level meetings that are scheduled. It would be nice to vary the schedule or if the information is important 
to have more than one call. 

ED introduces new points in conference calls and does not always provide written confirmation of new 
information. Calls, in particular, have had poor connections, interrupting conferences, etc. 

Ensuring that all conference calls provide a muted line. Most conference calls require participants to 
mute their own lines, which doesn't happen very often, making it extremely difficult to hear the message. 

Just the frequency of the use and also to allow more interaction. It would be better to divide states up to 
be more customer friendly and to allow for more interaction. 

Lots of other technologies available, e.g., podcasts; satellite conferencing; video streaming. Conference 
calls are not the only technology available. 

Not sure. 
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Q9 (OSEP). What can OSEP do over the next year to meet your state’s technical assistance and 
program improvement needs? 

Become more aware of the problems states face on a regular basis and remove the unreal expectations 
placed upon states. 

Continue the assignment of my current state contact. I have been through many state contacts in the 
past three years. While I know it is difficult to predict how long a person will remain in a position, I think 
some analysis of the rationale for the frequent turnover may be in order. Having to educate each contact 
assigned to the specifics on my state each time a new assignment is made is time consuming and 
frustrating, especially around the complexities of my state's assessment system. 

Cut back on the Indicator work. Working with OSEP is overly legalistic and far too often delayed 
responses just to check with OGC — this is supposed to be a service program, not a legal exercise. 

Expand Communities of Practice Continue efforts to enhance collaboration of technical assistance 
providers Implement electronic reporting for AT programs 

It is important for federal program managers to understand the unique needs of states. It is assumed that 
each and every state is unique but I can assure you that Alaska's schools are rural and remote unlike any 
other states definition of rural and remote. We would appreciate the opportunity to educate and provide 
learning experiences to our state contact. We believe that such opportunity only move to bring the state 
and federal government closer together on the issues that we might mutually encounter. 

Just be sure to read the time lines correctly. 

Less TA centers and more dollars to the RFtC's who care about meeting your individual needs and are 
responsive. 

Major Need: Improve the timeliness and consistency of information provided by senior staff to the SEA 
Directors. We still learn of 'what's going on' or 'going to happen' from other sources who are told these 
things in the context of other meetings well before we or our chiefs are given the information. This is not a 
good practice. 

Not change State contact; be responsive to requests; provide TA when requested; no additional data 
reports and reduce current reports; 

Over all I have been satisfied. Consistency of targets and know priority areas as soon as possible. 

Provide clarity on what requirements are. 

Provide more guidance around disproportionally and use of early intervening funds and reporting. 

Provide specific guidance on how to address underrepresentation data for SPP - provide clear guidance 
on suspension expulsion indicator - develop technical assistance center focused on bilingual special 
education - provide clear guidance regarding target population for 2 percent assessment 

Provide timely information 

Put into writing the information that is currently delivered only during teleconference calls, so that if the 
call occurs at a time the desired participant is unavailable, the information can still be obtained. 

Reduce non-mandated data collection requirements; make items more meaningful; cease changing data 
items with less than 2 years advance notice; provide greater technical assistance resources 

Spend less money on the large number of TA centers and give more to RRCs to provide specific state 
targeted TA. The centers are very difficult to work with and are all about publishing and name-dropping 

States more targeted technical assistance that goes beyond giving us Web site information. We need 
contact with specific activities that have worked in other states regarding student progress and 
improvement state indicator data. We need to get our feed back quicker. Your opportunity to work on the 
feedback gets here to late. By the time we get feedback the opportunity to act on it is already gone. 
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They need to be more responsive in a timely manner and provision of technical assistance online 
resources is not effective. 

We could use more assistance in helping to find quality technical assistance to bring to the state to assist 
districts in meeting targets on the SPP. 

With all of the regulations they should still make available e-mail address's to experts who will respond to 
specific questions that are not addressed in the technical assistance documents. 

Work with officials responsible for NCLD to promote consistent data definitions and reporting 
requirements. 

Q35. Finally, please describe how ED can improve its service to you. 

Technical Assistance/Guidance 

Again, I think targeted technical assistance that is specific to states needs. Technical assistance that 
results in improved data and improve students outcome. 

Provide clearer guidance; allow states more time to prepare to implement new data collection 
requirements 

Provide expectations and guidance up front before we are required to report instead of requiring a report 
on a nebulous item and then telling us we did it wrong. Keep reporting requirements to exactly what is 
required by law and no more. 



Consistency 

Consistency of liaison. We had four different ones in the past two years. 

Consistency in answers to questions about the collection and reporting of data for the State Performance 
Plans. Please note, the technical assistance offered in conjunction with the SIG project is outstanding. 

Just refer back to the previous comment in being more timely and consistent with information. 



Technology 

Continue to use technology to deliver information; Assure consistency in the assistance/information 
provided by staff to each state (i.e. SPP/ APR). 



Reporting Burden 

Decrease bureaucracy, paperwork, and allow states to identify 2 or 3 FAPE in the LRE Indicators to focus 
on and do not require all 8. Allow states to choose six general supervision indicators rather than all 12 to 
focus on — for the purposes of reporting. We understand we are responsible for compliance in all areas 
of the law, but the burden of reporting in the exact manner you wish, every year, on 20 Indicators is 
considerable. 

OSEP has required and changed so many things with the SPP that it is difficult to keep up. They seem to 
change data requirements and ignore the input from the public. The requirement for the parent indicator 
On the SPP is not founded in law anywhere and it costs us the most to implement and tells us very little. 

I hear a lot from/about the monitoring section and very little about the work of the research to practice 
division. What do they do ? 

Web site 

If there could be more links to physical questions and answers that would be greatly appreciated. 
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Communication 

Listen; take the time to understand the states issues when implementing so many different Federal 
programs that are not aligned. Figure out who is your customer. It is never clear who you believe is your 
audience. When given honest feedback it is often ignored because it does not match your mental model. 
Be respectful to the States. At the last meeting I attended there were a lot of new staff with the wrong 
attitudes. Most of the time it appears that they forget that this work is about serving the needs of children. 
They never report how they believe all their efforts are making the system better for infants, toddlers, and 
students with real data that their efforts are resulting in improvement. 

Provide opportunities for key stakeholders to provide input into ED's policies and mandates before 
implementation! ED could learn a lot by asking the people that have to implement their mandates! 

Read the information from my previous narrative within this survey. Additionally, understand the SEA is a 
catalyst for change and many times we are attempting to change systems and practices to improve the 
achievement of students with disabilities. Oftentimes, those who are needing to change most are those 
who are not reluctant to call USDOE. When OSEP leadership engages in conversations with individuals, 
such as higher education, the SEA deserves some professional courtesy. 



Timeliness/Responsiveness 

More timely responses to legal and regulatory questions; significantly simplify data collection 
requirements; increase cross agency collaborations -e.g., there is little evidence of policy discussions 
between OSEP and higher education where issues of students with disabilities should be a priority 

Provide answers in a more timely manner. Make reporting more user friendly, especially the Annual 
Performance Report. Eliminate redundant information submission between programs such as Title 1 and 
IDEA. 

Responsiveness is key; it must be timely, accurate, and relevant. 

Timely responsiveness to questions and issues raised. Listening to stakeholders and seeking input from 
stakeholders before setting policies that require more time to implement than states are given or that 
have consequences that were unforeseen by ED staff but could have been foreseen by state agency staff 
or other stakeholders had they been asked for input prior to issuing the policy. 

We are having a hard time getting any questions answered regarding EDEN/Ed Facts. We send emails 
and no one responds. How are we supposed to have accurate and timely data when no one will respond 
to our emails? 

Other 

My concerns, along with those of colleagues in other SEAs, have been shared verbally and in writing with 
senior staff in the US DOE OSERS/OSEP office; however, we have not filed a 'formal complaint' as we 
think it would be directly detrimental to our relationship to the OSERS/OSEP office and staff, which only 
serves to hurt the States. Retaliation is not uncommon. 



My staff and I are pleased with the support and encouragement that we receive from our State Contact. 
Any opportunity to provide better understanding regarding the provision of services in Alaska to our state 
contact would be appreciated. I would encourage OSERS/OSEP to allow staff to provide face-to-face 
technical assistance to states as well as the opportunity to gain insight and understanding about that 
provision of services through direct experience. 



None that I can think of. 
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OPEPD - EDEN/ED Facts Coordinators 

Q9. Please identify your state’s best example of collaboration across offices that you would offer 
as a model for ED. 

We have a data governance model that makes program areas responsible/accountable for the 
information they collect and report. It prompts collaborative responses to data quality issues to increase 
alignment across program areas and ensure the Department responds uniformly to LEAs on data issues. 



Q21. Please describe how ED could better use technology to deliver its services. 

Provide more WEBEX trainings and/or meetings to keep state staff up to date on best practices, future 
plans, etc. Use this technology to have monthly ED Facts 'town meetings' to keep stakeholders engaged 
and current on developments (EDFacts goals/plans/upcoming concerns, etc). 

Several of the meetings could be done via videoconference or Web ex, with more frequency and less 
travel. One face-to-face annual meeting versus 2 for example - both with NCES and IES Grantees. 



Q12 (EDEN/EDFacts). What has been the most significant change to your state data collection and 
submission process as a result of the EDFacts work? 

A vast increase in the detail of data we are required to report, (especially on small student subgroups 
such as migrant, ELL, and homeless) has prompted a rapid transition from legacy aggregate collections 
to student-level collections. Far more program area staff have had to become involved in data 
submission. 

Attention to data accuracy and organization of federal reports. 

Consolidation of data sources. Consolidation of identification sources. 

Consolidation of some Federal Collections into EDFacts. 

Data collection has changed very little to date, but will increase as a result of EDFacts’ more detailed 
requirements. The submission process is more complex, detailed, and time consuming than prior to 
EDFacts. Data that was reported as a single number in the CSPR is now an EDEN file with many new 
data elements. Submission work has increased significantly. 

Data quality from the LEA to the SEA has improved. More collaboration within the SEA has also been a 
result. 

EDFacts has driven us to look much deeper at our data collection systems, data elements collected, 
interconnectivity of our datasets, QA processes, and take steps to improve in all aspects how we view 
and handle data. 

Finding out who submits what and how it is collected. Trying to explain why information is collected 
through EDFacts and reconcile differences in reporting. 

Higher standards. Reduction in redundancy. Better and more consistent data. 

in the short term has added another layer of processing/development, etc and has been very resource 
intensive. We are hoping that in the long-term we will see some benefits. It has shifted responsibility in 
reporting from several teams within the department to a single team... and will make reporting more 
centralized. 

Increase in EDFacts submissions due to work with USED. 

Intra-departmental communication has improved, and more offices are showing appreciation of each 
others efforts. Collection redundancies are being identified and managers are designing more efficient 
data collection guidelines. 

It has become more automated. 
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It was the catalyst for us to implement Data Steward and Data Governance programs. These programs 
have since been important to our work with master data management and data warehouse initiatives. 

More coordinated with better timelines. 

More coordination and collaboration between program offices at state level and between Michigan and 
USED. 

More coordination between programs. 

More detailed attention to the data submitted. 

More process, more processed collections and data submissions. 

My job has changed 100%. 

Not very much has changed for us. 

Significantly increased the whole staffs workload. 

Streamlined/reorganized the submission plan 

The EDFacts requirements are helping us better understand our data collections and metadata. 

There are a lot more data checks. 

We have been able to eliminate unnecessary items from our data collections. 

We just received an Ids grant. With out the help the questions would have been answered 2 or 3 instead 
of 8 or 9 the longitudinal data system grant. 

We've had to spend a lot more time and effort on data collection and changes to what is required to 
report. 

While I believe in the long run, EDFacts will be a good thing, the data collection/submission 
responsibilities have significantly increased the workload for our Data Management staff. We are a small 
state with limited resources and at times find it difficult to hire people with the technical qualifications 
needed. 

Work has been directed from the program offices to the data management shop without commensurate 
funding. Program offices do not yet trust data management's reports, so double the work has to be done 
to prove the results are correct. EDFacts is not relevant to our data collection process. 



Q13 (EDEN/EDFacts). How can the Department’s EDFacts team be most helpful to you in meeting 
federal mandates for data collection, submission analysis, and reporting in the coming year? 

1) Voice to ED our need for additional staff/funding resources to accomplish the HUGE task of federal 
reporting; 2) Continue to enhance and improve the tools (Submission Plan spreadsheet, EDFacts Portal 
elements, etc.) 

Additional resources would be great. 

Be more sensitive to the fact that many of use have silos of data that exist in many formats. We are not 
always able to send in the files in the formats requested, as we do not have the resources. The files 
should be acceptable in other formats until we have the capabilities to submit them in the EDFacts format. 

Better communication to various programs about the collections. 

Clarity and consistency of data definitions. 

Clear and concise communications. 

Continue to be flexible in negotiating data submission timelines with states. 

Continue to provide timely information regarding reporting requirements and changes 
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Crystal clear guidelines and file specs for data collection. 

Eliminate technical glitches in the EDEN system, e.g., EDEN-DANS discrepancy analysis; eliminate the 0- 
filling requirement in EDEN files; provide more on-line resources, e.g., re-introduce the Charter School 
report; start allowing states to use more of the EDEN data for reporting needs a la CCD's table building 
functionality. 

Give states funding to buy tools, such as software and training; provide us with online metadata tool; and 
facilitate establishment of user groups for states with like systems. 

I find the trainings and the meetings very helpful, so more continuation of those. 

I really need some assistance in staffing here. 

I think they are doing a pretty good job as they are. 

It would help to have more emphasis and reference to submitted data. Offices tend to be less resistant 
when they see value to collection rather than just for the sake of it. 

Keep changes to a minimum. 

More one-on-one training with state staff responsible for this process. 

More support in terms of federal dollars. A full time coordinator position for each state. The full time 
coordinators capacity for. More coordination of our federal program efforts offices federal state. 

More timely of the materials received from ED. We still don't have all the file specifications yet we are 
required to tell what we can and cannot submit for the coming year. We receive materials from ED (like 
IDEA congruency and submission/transition plan) with little turn around time for us. The IDEA 
congruency we have less than two weeks to get that figure out and back to ED. We don't have enough 
staff to do all this. 

On-going funding. Changes to submission plan have been useful ( with expected dates of when data is 
first used). 

Please fund an EDFacts coordinator for each state. Please continue to work with federal program offices 
to continue to refine data needs. Please examine the utility of the data collected for evaluation and 
reporting. 

Provide at least two years lead time for the introduction of new data groups or elements before they 
become mandatory; fund the EDFacts Coordinator position; provide the specifications at least three 
months prior to the collection deadline; resolve the race/ethnicity implementation process questions and 
differences between US ED offices. Provide SEA data quality training courses onlinejsimilar to the LEA 
courses now offered by CCSSO, SIFA, and Central Susquehanna Intermediate Unit). 

Provide realistic lead-time to complete tasks. Improve communication of known issues. Improve 
communication regarding use of data by federal program areas. Communicate long-ranges plans and 
goals. Eliminate surprises. Provide more frequent EDFacts status updates. Explore ways the EDFacts 
team and state staff can have direct dialogue. Continue to seek funding to provide to states. 

Providing the funding we have been promised over the last two years and have never received. 

Releasing changes and file specifications earlier. Developing more reports that coordinators can run from 
the EDFacts Reporting site. 

The EDFacts team needs to initiate and continue conversations with US DOE program area offices 
regarding the quantity and especially the data categories/permutations required by each file specification. 
These discussions should ensure that only data categories and permutations of these data categories 
that US DOE program offices are actively using to make programmatic and policy decisions should be 
collected. Often, the data categories are the reason we cannot submit a complete file. 

To keep the communications channel clear. 
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Q35. Finally, please describe how ED can improve its service to you. 

Accessibility 

Better access to the data provided by our state (for data quality review purposes). 

The ED EDFacts staff members are exemplary in their professional knowledge, support, and assistance. 
Provide multiple access licenses to the EDFacts reporting system on a free or fee basis. 



Web site 

I use the ED Web site often to reference material for policy, and referral for Standard Practice. It would 
help to have more advanced features in Ed Search Engine. Being able to provide efficient searches by 
exact phrases would make my research less time consuming. 



Responsiveness 

Continue to be responsive and give weight to requests from the State Coordinators. ..As you have done in 
the past. 



Reporting Burden 

Extended timelines from when we receive materials and when they have to be completed and returned to 
ED. Also, less changes in the reporting requirements. We are scrambling every year to change our data 
systems to meet yet another ED requirement. 



Documentation 

In getting the documentation data documents out earlier. 

They could streamline the reporting system. More consistent documentation. The program offices do a 
better job of sending about any projects information about what’s going on. 

To ensure States can plan for and fulfill the requirements of EDFacts, ED must release the file 
specifications for the entire school year of files in advance of the school year. In addition , there needs to 
be greater collaboration and uniformity in reporting areas that cross programs (e.g., Highly Qualified, A YP 
reporting) so States don't have to report the same type of information in numerous ways. That said, the 
EDFacts initiative has prompted positive change in many SEAs regarding data collection, quality, and 
reporting. 



Training 

In the area of EDFacts, we would like to see more one-on-one training. With limited staff and resources it 
is hard to send our people to national trainings/conferences. 

Funding/Resources 

Provide more funding for EDFacts to the states. 

Provide more tools, funds to purchase tools you can't provide, and access to national conferences. 
Provide staff assistance here at the State level. 
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Consistency 

Speak one message with one voice, no matter what program area is speaking. Recognize that just 
because ED says so, doesn't make it so here in the trenches! Provide funding to allow states to provide 
AT LEAST 1.0 FTE of staff dedicated exclusively to EDFacts Reporting duties. 



Other 

I am pretty happy with what they are doing, so keep doing what there doing. 
I can't think of anything off the top of my head. 

I only deal with ED on EDEN/EDFacts and have no complaints at this time. 

No recommendations. They do a good job. 

No I cannot think of anything specific at this time. 
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OVAE - Career and Technical Education State Directors 

Q9. Please identify your state’s best example of collaboration across offices that you would offer 
as a model for ED. 

We work in teams on most things with representatives from secondary, postsecondary, data, finance, 
compliance and instructional programs - folks all work together on projects so all perspectives are taken 
into consideration. 



Q21. Please describe how ED could better use technology to deliver its services. 

Provide electronic application completion option; hold Webinars rather than conference calls; hold live 
interactive TAV meetings; provide Web-based training regarding FAUPL definitions. 

Q1 1 (OVAE). What can OVAE do over the next year to meet your state’s technical assistance and 
program improvement needs? 

Be clearer up front about interpretations of legal limitations before field-based work groups spend 
valuable time developing strategies that are later determined to be non-compliant. I recognize this 
requires getting the OAG to cooperate, but it would be very respectful of efficient use of limited resources. 

Be flexible - we are working toward requirements, but it will take a while to get there. 

Continue collaboration with NASDCTEC and NAPE, and continue use of DTI, MPR, and AED as 
technical assistance providers. 

Continue to provide opportunities for one-on-one TA visits and conference calls. 

Continue to support state implementation of Perkins IV - with flexibility yet consistency as appropriate for 
individual states. 

Guidance on Technical Skill Assessment 

Improve the CAR reporting system by having us report later in the school year 

Just continue to have patience with us as we work through the transition from Perkins III to Perkins IV, 
especially in the Technical Skill Attainment area. With the ability to sanction states administrative funds if 
they do not meet Performance Standards and changing how things are to be measured on those 
Standards, states must be given a grace period for the change. The OVAE staff that I have worked with 
have been outstanding and very helpful. I always feel they are there to help! 

Maintain consistency in staff that are assigned to a state. Provide second and third lines of support when 
the primary contact cannot be reached. Acquiring immediate feedback to urgent issues is usually 
impossible. 

More information involving Programs of Study. Collect and disseminate, on a regular basis, updates as to 
how states are doing in their efforts to create Technical Assessments. 

Our primary focus of need is in student assessments and in development of programs of study. 

Provide clarity on definitions, esp. skill attainment 

Provide more extensive guidance and support for technical skill attainment. 

Share best practices as determined through OVAE visits to states 

We suggest that monthly (or as needed) videoconferences are scheduled to keep states informed about 
key issues, and provide guidance on new projects and work expectations. 
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Q35. Finally, please describe how ED can improve its service to you. 

Funding/Resources 

Advocate for additional funding for our programs. 

Please make sure that the Research Center is given proper support for their projects. They do a great 
job. The research is very relevant to what we are doing on a daily basis. Overall, ED is very customer 
oriented and is doing an effective job. By way of clarification, I almost never would give a 10 on this type 
of survey, so 8 is close to my highest rating. 



Consistency/Accessibility 

As previously stated: States need a Single Point of Contact for their technical assistance needs. It is 
usually an ever-evolving list of staff; always under revision and reorganization. Furthermore, when an 
issue or a question requires an immediate response there is no (understood) means of accelerating an 
issue. Typically, on a Friday afternoon, NO ONE is available to answer the phone or respond to an email 
at OVAE despite the time differences that states are functioning within. All this said: CONSISTENCY 
AND ACCESSIBILITY are the themes intended to be conveyed. 

Not be inconsistent with responses. Improve the Website. It is impossible to find anything you are 
looking for without spending hours going through old postings. Categorize things better 

Please make certain that whenever we call, there is at least one staff member available to respond so we 
do not have to leave voice messages that are returned 3-4 days later. That is too late. 

Provide consistent information on FAUPL definitions; formalize the negotiation process, allow for 
attainable percentage targets. 



Reporting Burden 

We need more lead time when decisions are made on collecting data. It takes a minimum of two years if 
we are making changes in data collection at the state level and sometimes it takes even longer at the 
local level. In order to be effective at the state and local level, we need definite decisions and ample time 
to implement requirements for standards. 



Communication 

Provide greater clarity in the email and paper correspondence; many times we have to sift through the 
government jargon to get to the real information. 

Some additional clarification under Perkins IV - technical assistance 

Training 

Continue to provide 'best practice' information about secondary and postsecondary Perkins-funded 
programs from other states; give us frequently updated, quality information on 'gold standard' 
certifications; make all workshops and conference sessions available via Webinars. 



Timeliness/Responsiveness 

Probably the only issue to resolve is returning phone or email questions in a more timely manner. 
Quicker turnaround response to states 
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Other 

Develop an alternative to OMB's PART system, which is deeply flawed and useful for very little except 
political manipulation 
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OVAE - Directors of Adult Education and Literacy 

Q9. Please identify your state’s best example of collaboration across offices that you would offer 
as a model for ED. 

Collaboration WIA WITH HIGHER EDUCATION. 

Implementation of a bridge to employment program between and cooperation on costs (English literacy 
funded by ABE and employment related preparation classes and experiential learning activities funded by 
WIA). 

Shared Youth Vision Project 

The MN Adult Ed. office is collaborating well with the MN College office and the MN Labor office on the 
implementation of a Joyce Foundation transition to post secondary grant. 

Q21. Please describe how ED could better use technology to deliver its services. 

Make Web site more user-friendly 

National Directors' meetings; Webcast of these would allow state directors to set up appropriate state 
groups to be involved in relevant sections of these meetings and to have facilitated discussions at the 
state and local level. 

Teleconferencing to replace annual meetings. 



Q12 (DAEL). What can DAEL do over the next year to meet your state’s technical assistance and 
program improvement needs? 

1. Share sample policies and program descriptions that are contributed by states around certain high- 
profile topics e.g. - distance education policies. 2. Share actual 'findings' text from federal monitoring 
reports - remove the state names from the document first. 

Become more customer-service oriented. Currently OVAE's tenor is regulatory and punitive. 

Continue the practice of inviting state ABE directors to help plan the annual meetings. 

Continue the training in all areas; share the results from initiatives that limit participation due to funding, 
help with training that will be needed for the new series of the GED test for our professional developers to 
teach our teachers like we did before in 2002 in Florida. Keep up the high quality and standard you have 
set. 

Continue to do what they have been doing. Also, try to convene regional meetings for state directors. 

Continue to offer online resources, online trainings to assist if meetings cannot be attended, face-to-face 
meetings and continued technical support. 

Current legislation information about act. Reauthorization, assistance with performance measures and 
more advanced knowledge new requirements. 

Designate a single person to do a single training for all of the states an all of the regionals, so a regional 
training would be a good idea. 

Determine how to provide assistance to small states as well as the larger states. It seems that the focus is 
often on the large states. 

ESL-ABE transition 

Help with teaching writing. 
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I think that it would be worthwhile for OVAE to construct a model funding formula for states to consider. I 
think states would welcome a general not too complex of a formula to consider and build on based on 
each states individual needs. ..of course this would be optional. 

It would be nice to have regional meetings; at which we could discuss priorities and needs. 

More assistance in the teaching of math and continued support in transitioning students from GED to post 
secondary 

None at this time 

Offer at a low cost. 

Provide funding to states that are participating in national activity projects as a pilot state. 

Provide section in Thursday Notes on helpful tips from OVAE staff regarding elements of accountability or 
policy issues - updates on national projects. 

Provide some guidance on the transition of ESL students to ABE services 

Provide state specific results and outcomes in regards to performance based funding, content standards 
implementation, transition to post-secondary, and partnership-building with Title I (one-stops). Has 
student performance improved as a result of these efforts? 

Publish a policy and procedures manual. 

Standards and curriculum align 

Technical assistance is great. Wyoming is having trouble with the data provider and the field 
understanding what is required. 

They need to work on the issue of the sixty-hour requirement in the testing. It shouldn't be that high. 

There are policies being put into place without any national research to verify the validity. 

To make the Web site more user friendly. To add additional information and to find information. 

Well need lots of assistance with a new State Plan whenever reauthorization occurs. 



Q35. Finally, please describe how ED can improve its service to you. 

Documentation 

A policy manual would be very helpful. 

Develop a prototype RFP. 

Timeliness/Responsiveness 

A timely response and accurate information. It needs to be unambiguous. 

Regular updates on special projects; timely dissemination of resources to those not in the projects. 

Respond quickly when we have a question or a concern. Have them do more research on the western 
region programs or have them be better informed as to what the adult education needs are. 

Meetings/Training 

It would be nice to hold more Webinars and not have to travel as often to face-to-face meetings. It is hard 
to get out of state for the meetings but I need the information. 

More facilitated opportunities to communicate with other state directors would be helpful. 
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Provide some training through distance learning formats since travel is being curtailed due to budget 
concerns and reduction of state funding. 

We do some more work on distant learning. More effort in distant learning. 



Communication 

Proactively issue policy briefs to provide further clarification to areas of the legislation where there are 
frequently asked questions. 

Work as closely as possible with the National Council of State Directors of Adult Education, our advocacy 
group. 



Consistency 

Provide consistency and clarity in definitions and expectations of standardization between states 
regarding the interpretation/clarity of definitions and expectations. 



Other 

Be an advocate for the re-authorization of WIA to be more amenable to the current societal needs. 

ED has provided satisfactory services, and is prompt in responding to questions and/or concerns. 
Sometimes the responses are not we expect to hear as ED is far behind with the political arrangements of 
the Freely Associated States with the U.S. Government. ED should also comply with the AEFLA as they 
expect grantees to do so. 

ED is doing the best they can with the resources available. Thank you. 

Just continue doing as they have been doing in the past. They return phone calls and emails very 
promptly and I appreciate that. Are always friendly and appear glad to assist us. 

Replace the DAEL Director, [name deleted]. Replace punitive and arrogant attitude with one of customer- 
service. 

Services at all levels have been very satisfactory in my 6-month tenure as a state director. 
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APPENDIX E: Explanation of Significant Difference Scores 
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The charts depicted throughout this report compare 2008 to 2007 scores and note significant differences. 
The following provides some background on how CFI calculates and reports significant differences. 



Whether a significant difference exists between two scores (mean scores reported on a 0 to 100 scale) 
depends on the sample size, the standard deviation and the level of significance selected. CFI employed 
a 90% level of confidence to check for significant difference on all questions. This is the standard level 
used in most of our studies. However, standard deviation and sample size vary from question to question. 
Therefore, some questions may show a small difference in scores as being significant, while others show 
a much larger difference not being significantly different. 



In CFI’s studies standard deviation, which is a measure of how dispersed scores are around the mean, 
typically ranges from 15 to 30 points for any given question as reported on a 0 to 100 scale. A higher 
standard deviation results in a larger confidence interval around a score, so a larger difference in scores 
would be required to be significant. To further illustrate how the dispersion of scores affects significance 
testing between two sets of scores, two scenarios are provided. Assume 350 responses were collected in 
both year one and year two and a 90% level of confidence is used. In the first instance, the standard 
deviation is 15 points in both years, so scores were fairly uniform without much dispersion around the 
mean. In this case, a difference in scores between years one and two of less than 2 points would be 
significant. However, if the standard deviation were 30 points instead of 15 in both years, so scores were 
not as uniform and much more dispersed around the mean, nearly a four-point (3.7) difference in scores 
between years one and two would be necessary to be significant. 



With respect to sample size, larger sample sizes result in smaller confidence intervals. Thus, larger 
sample sizes require smaller differences in score to be significant. At a program-level, State Title I, Title III 
State Directors, State Education Technology Directors, Directors of Adult Education and Literacy, and 
Career and Technical Education State Directors had more responses and larger sample sizes. 
Conversely, Chief State School Officers, State Title V, Part A Directors, and Lead Agency Early 
Intervention Directors had lower response rates and smaller sample sizes. 
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